View
4
Download
0
Category
Preview:
Citation preview
Western Michigan University Western Michigan University
ScholarWorks at WMU ScholarWorks at WMU
Dissertations Graduate College
12-2004
Common Sense Parenting (CSP) Learn at Home Kit: A Clinical Common Sense Parenting (CSP) Learn at Home Kit: A Clinical
Effectiveness Evaluation of a Commercially Available Video Effectiveness Evaluation of a Commercially Available Video
Training Program for Parents Training Program for Parents
Sean T. Smitham Western Michigan University
Follow this and additional works at: https://scholarworks.wmich.edu/dissertations
Part of the Child Psychology Commons
Recommended Citation Recommended Citation Smitham, Sean T., "Common Sense Parenting (CSP) Learn at Home Kit: A Clinical Effectiveness Evaluation of a Commercially Available Video Training Program for Parents" (2004). Dissertations. 1141. https://scholarworks.wmich.edu/dissertations/1141
This Dissertation-Open Access is brought to you for free and open access by the Graduate College at ScholarWorks at WMU. It has been accepted for inclusion in Dissertations by an authorized administrator of ScholarWorks at WMU. For more information, please contact wmu-scholarworks@wmich.edu.
COMMON SENSE PARENTING (CSP) LEARN AT HOME KIT:A CLINICAL EFFECTIVENESS EVALUATION OF ACOMMERCIALLY AVAILABLE VIDEO TRAINING
PROGRAM FOR PARENTS
by
Sean T. Smitham
A DissertationSubmitted to the
Faculty of The Graduate Collegein partial fulfillment of the
requirements for theDegree of Doctor of Philosophy
Department of Psychology
Western Michigan UniversityKalamazoo, Michigan
December 2004
COMMON SENSE PARENTING (CSP) LEARN AT HOME KIT:A CLINICAL EFFECTIVENESS EVALUATION OF ACOMMERCIALLY AVAILABLE VIDEO TRAINING
PROGRAM FOR PARENTS
Sean T. Smitham, Ph.D.
Western Michigan University, 2004
Much has been made of the gap between psychotherapy research and clinical prac-
tice. Most current psychotherapy research is focused on what could be viewed as
macro-level efficacy type issues, while practicing clinicians are often most concerned
with micro-level effectiveness questions. The current study - an evaluation of a
parent training (PT) program provides an example of how scientist-practitioners
can contribute meaningfully to psychotherapy research by conducting small scale
clinical effectiveness studies. Parent Training (PT) is a well established efficacious
treatment approach for child disruptive behaviors and non-compliance. Recent re-
search has also established that self-administered videotape PT programs may also
be efficacious. A popular, commercially available, self-help video training program
for parents – the Common Sense Parenting: Learn-at-Home Kit published by Boys
Town Press – was the subject of this study. The results indicate that the CSP Learn-
at-Home Kit was largely ineffective in reducing child disruptive behaviors. Results
are discussed in the context of the efficacy/effectiveness distinction and the clini-
cal research/clinical practice gap. Hypotheses about why the CSP Learn-at-Home
program may be minimally effective are provided.
UMI Number: 3162405
31624052005
UMI MicroformCopyright
All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company 300 North Zeeb Road
P.O. Box 1346 Ann Arbor, MI 48106-1346
by ProQuest Information and Learning Company.
Copyright by
Sean T. Smitham
2004
Acknowledgments
The completion of this project would not have been possible if it were not for the
combined commitment, support, and encouragement of a number of individuals.
First and foremost I’d like to acknowledge the love and support of my family and
friends: Mikael Madison, Spencer Luke, my Beloved Shea, Jim and Jackie Smitham,
Jon and Jennifer Smitham, Dana Smitham, Floyd and Freda Stewart, the Sieckman
and Stergar clans, Stephanie Kerbel and her family, Patrick Mulick, Emily and Jon
Fredricks, Kathryn and Troy Bell, Tara Cornelius, Blake Lancaster, Laura Murray,
Jeff and Cari Porter, Kim and Chris Snook, the students, faculty and staff of Western
Michigan University Psychology Department. Without you, none of this would be
possible.
Special thanks to my committee members: Kristal Ehrhardt, Linda LeBlanc,
and Roberto Flachier for sharing their time, talents, and wisdom with me. Your
guidance, professionalism, flexibility, and expertise were greatly appreciated. Linda
Rowen and Donna Areaux went above the call of duty to ensure I met all admin-
istrative requirements and deadlines. Thank you all for helping me maintain my
integrity and sanity during the final stages of this project.
I’d also like to acknowledge the staff of the Specialized Clinical Services
and Research department at Girls and Boys Town including: Clint Field, Mike
Handwerk, Julie Almquist, Mike Axelrod, and Pat Friman for their support and
guidance. With gratitude, I especially want to thank those families that participated
ii
in the study for their time, effort, and patience. May you be blessed with happiness
and good health.
Finally, I’m eternally grateful for being blessed with an amazing mentor,
advisor, and friend in Galen Alessi. Galen, I simply don’t have the words to express
the depth of my gratitude or admiration. I’m forever in your debt for your continued
investment in my development both as an individual and as a professional. I look
forward to many years of consultation and conversation with you.
Sean T. Smitham
Western Michigan University
December 2004
iii
Contents
Acknowledgments ii
List of Tables vii
List of Figures viii
Chapter 1 Introduction 1
Chapter 2 Literature Review 3
2.1 Historical Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Parent Training for Non-compliant Children . . . . . . . . . . . . . . 5
2.3 Training Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4 The Boys Town Model and CSP . . . . . . . . . . . . . . . . . . . . 8
Chapter 3 Methods 10
3.1 Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2 Subjects and Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.3 Recruitment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4 General Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.4.1 Phase 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.4.2 Phase 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.5 Independent Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.6 Measures/Dependent Variables . . . . . . . . . . . . . . . . . . . . . 14
3.6.1 Parent Daily Report (PDR) . . . . . . . . . . . . . . . . . . . 14
3.6.2 Eyberg Child Behavior Inventory (ECBI) . . . . . . . . . . . 14
3.6.3 Consumer Satisfaction Survey (CSS) . . . . . . . . . . . . . . 15
iv
3.6.4 Family Data Questionnaire (FDQ) . . . . . . . . . . . . . . . 15
3.6.5 Parenting Stress Index - Third Edition (PSI) . . . . . . . . . 15
Chapter 4 Results 16
4.1 Subject 03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.2 Subject 05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.3 Subject 02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.4 Subject 01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.5 Subject 06 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.6 Subject 08 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.7 Subject 11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.8 Subject 10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.9 Visual Inspection Summary . . . . . . . . . . . . . . . . . . . . . . . 23
4.10 Clinical Significance . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.10.1 Reliable Change Index . . . . . . . . . . . . . . . . . . . . . . 25
4.11 Social Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Chapter 5 Discussion 30
5.1 More Than Common Sense . . . . . . . . . . . . . . . . . . . . . . . 30
5.2 Efficacy vs. Effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.3 Clinical Research versus Clinical Practice . . . . . . . . . . . . . . . 36
Chapter 6 Conclusion 40
References 43
Appendix A Figures 50
Appendix B Tables 65
Appendix C Consumer Satisfaction Survey - Comments 71
C.1 What tips and suggestions from the videotapes did you attempt to
use in your home? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
C.2 What tips and suggestions did you find most helpful? . . . . . . . . 72
C.3 What tips and suggestions did you find least helpful? . . . . . . . . . 73
v
C.4 What suggestions would you have to improve the videotape training
system? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
C.5 Would you recommend the ”Common Sense Parenting Learn-at-Home
Kit” to a friend or family member who was having difficulty managing
their child’s behavior? Why or why not? . . . . . . . . . . . . . . . . 74
C.6 On a scale of 1 (very unsatisfied) to 10 (very satisfied) how would you
rate your experience as a participant in this study? . . . . . . . . . . 74
Appendix D Parent Daily Report (PDR) Items 76
Vita 78
vi
List of Tables
B.1 CSP Learn At Home Video Sessions . . . . . . . . . . . . . . . . . . 66
B.2 PDR Means by Subject for Baseline and Treatment Conditions . . . 66
B.3 ECBI Raw Scores Pre- and Post-Treatment . . . . . . . . . . . . . . 67
B.4 PSI Raw Domain Scores Pre- and Post-Treatment (Subjects 1-6) . . 68
B.5 PSI Raw Domain Scores Pre- and Post-Treatment (Subjects 8-11) . 69
B.6 PDR Reliable Change Index (RCI) . . . . . . . . . . . . . . . . . . . 69
B.7 ECBI Reliable Change Index (RCI) . . . . . . . . . . . . . . . . . . . 70
B.8 PSI Reliable Change Index (RCI) . . . . . . . . . . . . . . . . . . . . 70
vii
List of Figures
A.1 PDR Data Cohort 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
A.2 ECBI Data Cohort 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
A.3 PSI Data Cohort 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
A.4 PDR Data Cohort 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
A.5 ECBI Data Cohort 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
A.6 PSI Data Cohort 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
A.7 PDR Data Cohort 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
A.8 ECBI Data Cohort 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
A.9 PSI Data Cohort 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
A.10 PDR Total Behavior Score RCI . . . . . . . . . . . . . . . . . . . . . 60
A.11 ECBI: RCI T1 to T2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
A.12 ECBI: RCI T2 to T3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
A.13 ECBI: RCI T1 to T3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
A.14 PSI: RCI Total Stress and Parent Domain . . . . . . . . . . . . . . . 64
viii
Chapter 1
Introduction
”At present, clinical research has little or no influence on clinical practice. This
state of affairs should be particularly distressing to a discipline whose goal over the
last 30 years has been to produce professionals who would integrate the methods
of science with clinical practice to produce new knowledge. (p.147)” - D.H. Barlow
(1981)
Despite the prolific output of psychotherapy research over the last 50 years,
the clinical activities of most psychotherapist remain largely unaffected by findings
from empirical research. However, there is a growing number of psychotherapists
trained as scientist-practitioners who provide direct clinical care. They ply their
wares in the gap that exists between clinical research and clinical practice – some
surviving, some thriving amidst the creative tension between psychotherapy research
and psychotherapy practice. Many, like the author, are interested in applying their
training not only in the service of providing the most effective treatment possible for
clients, but also in contributing meaningfully to the scientific and philosophical dis-
course surrounding the art and science of therapy. The current study is an attempt
to meet both goals by serving as: a) a model for the type of effectiveness evaluations
that may be feasible for busy clinicians and b) a catalyst and data point for further-
ing the discussion concerning the relationship between psychotherapy practice and
psychotherapy research. The related domains of child conduct problems and parent
training provide the context for this investigation.
Conduct problems in children (including minor compliance problems like
whining, tantrums, backtalk, and failure to follow adult instructions) are one of the
1
most frequent reasons for clinical referrals in child and adolescent treatment service
settings accounting for one-third to one-half of referral cases in those settings. One
of the most effective treatments for child conduct problems is Parent Management
Training (PMT) or simply Parent Training (PT).
Parent training is traditionally provided through individual consultation with
a therapist or a group training format. Both training formats have utilized video-
tape technology - either to provide specific performance feedback to individuals
or illustrate correct and incorrect use of parenting skills. However, training par-
ents using videotape technology in this manner is quite expensive as it requires a
trained professional to provide specific coaching during the review of the tape. Re-
cent research has established the efficacy of a self-administered videotape modeling
training program for parents; however, the program is still rather expensive and it
is not widely available to parents. By contrast, there are a number of commercially
produced, reasonably priced parent training videos that profess to provide effective
parenting strategies and incorporate videotape modeling of these strategies. These
”self-help” parenting tapes are readily available and relatively inexpensive, yet very
little research has been conducted on the effectiveness of these programs. A sys-
temic evaluation of the effectiveness of these programs would seem valuable given
the increasing need for efficient and effective parenting interventions, driven by the
disturbing observations that many parents who could benefit from parent-training
services never seek services, never follow through in obtaining those services when
they are recommended, or terminate training early. This study attempts to eval-
uate the effectiveness of a commercially produced, self-administered video training
program associated with a popular self-help parenting book - ”Common Sense Par-
enting” (CSP) published by Boys Town Press. Thus, some of the same outcome
measures used in previous PT efficacy studies are used in this effectiveness trial.
2
Chapter 2
Literature Review
Conduct problems exhibited by children are currently the greatest social concern
faced by parents and caregivers in the United States (McMahon & Wells, 1998).
Defiant, oppositional, and aggressive behavior problems comprise the most frequent
referrals for clinical and school based services - accounting for one-third to one-half
of clinic referrals for children (Kazdin, 1987). Research has shown that aggressive
and non-compliant behaviors observed for young children at home predict similar
problems at school and tend to remain stable over time (Reid, 1993). Moreover,
young children who engage in high levels of aggression across multiple settings have
an elevated risk of developing persistent behavioral, academic, emotional, and social
problems later in life (Miller & Prinz, 2003). Children who display early-onset con-
duct problems are at greater risk for long term negative consequences that include
substance abuse, unemployment, marital problems, divorce, car accidents, physical
problems, and welfare dependence (Kazdin, 1987; Reid, 1993). In addition, un-
manageable conduct and related problems are associated with a large and growing
population of youth being placed in out-of-home care, the most common of which is
group residential care (Friman, 2000). The prolonged exposure of youth displaying
pervasive and persistent conduct problems to special assistance from school, mental
health, and criminal justice systems results in tremendous societal cost (Kazdin,
1987; 1997).
3
2.1 Historical Overview
Behavior Modification, the broader discipline that eventually gave rise to Parent
Training, developed in the 1920’s, flourished for a short time, and then was driven
underground by the ascendance of psychoanalysis. The discipline reemerged in the
early 1960’s due in large part to its notable success in treating what were considered
intractable problems (Graziano & Diament, 1992). Parent Training marked a pro-
found shift from the traditional, psychoanalytic one-to-one, therapist-client model
of care that dominated psychotherapy at the time. This paradigm shift was driven
in large part by the growing need for child treatments that were more successful
than traditional psychodynamic approaches.
Psychologists in the late 1960’s and early 1970’s were alarmed by the large
number of children in need of services. It was well understood that in order to
change behavior reliably, and maintain that change from day-to-day, interventions
frequently had to occur in the natural environment of the subject. Researchers rea-
soned that only teachers and parents had sufficient numbers and access to deal with
the problems of children (O’Dell, 1974). This line of reasoning was further bolstered
by the findings of a young researcher by the name of Gerald Patterson and his col-
leagues (Patterson, Littman, & Hinsey, 1964). After studying children displaying
disruptive behaviors, Patterson and colleagues concluded that contingencies in the
child’s social environment, rather than internal psychological traits, were most re-
sponsible for the child’s adjustment problems. They suggested that retraining the
child’s parents may not only be desirable but often absolutely necessary.
Patterson published the first widely used parent training book in 1968 (Pat-
terson & Gullion, 1968). The following decade saw the emergence of a new kind of
self-help book focused on the challenges of parenting (e.g., Patterson, 1971; Christo-
phersen, 1982). Since then, a steady stream of parenting books has proliferated the
self-help market (Latham, 1994; McIntire, 1996; Forehand & Long, 1996; Gottman
& DeClaire, 1998; Christophersen & Mortweet, 2003).
Incorporating the parent as the change agent was intended to provide im-
proved access to the child’s natural environment; more reliable and valid informa-
tion; better generalization, maintenance, and prevention; and improved cost effi-
ciency (see Graziano & Diament, 1992, for review). Once researchers demonstrated
that parents could learn behavioral principles and apply them to change the behav-
4
ior of their children, they began to investigate the effectiveness of PT in ameliorating
a variety of child behavior problems and focused on the most effective and efficient
means of training parents. PT has been applied to non-compliant children (Fore-
hand & King, 1977), children who are developmentally disabled (Hudson, 1982),
hyperactive children (Bor, Sanders, & Markie-Dadds, 2002), children with eating or
medical problems (Bigelow & Lutzker, 2000), and children with specific behavioral
problems (Forgatch & Toobert, 1979). The following sections, while not an exhaus-
tive review, discuss key findings in the use of PT to treat childhood noncompliance
and the use of various training formats to teach parents new skills.
2.2 Parent Training for Non-compliant Children
While there is no universal standard parent training protocol, many of the most
effective and empirically supported versions target similar parenting behaviors. Dif-
ferent parent training packages differ primarily in where they place their emphasis.
Parenting behaviors frequently emphasized are: giving effective directions, noticing
and rewarding good behavior, using non-coercive discipline, monitoring child ac-
tivities, communicating about emotions, and problem solving. While more recent
research has focused on validating the efficacy of different parent training packages,
early parent training research focused on evaluating the effectiveness of its various
components. Knowledge of key findings in this area is valuable when assessing the
relative strengths and liabilities of current parent training packages. The following
paragraphs provide an overview of important discoveries in early parent training
research.
Most parents seek parent training in order to reduce child disruptive and/or
non-compliant behaviors. Early research established that training parents to use
time-out (TO) and extinction procedures proved effective in reducing child non-
compliance (Day & Roberts, 1983;Hamilton & MacQuiddy, 1984;Hobbs, Walle, &
Caldwell, 1984;Roberts, 1988;Walle, Hobbs, & Caldwell, 1984). These findings flew
in the face of a pervasive popular belief among parents that physical punishment
is more effective than other forms of punishment in decreasing problem behavior.
In fact, research demonstrated that the addition of a spanking contingency is no
more effective than a simple barrier or a TO room (Day & Roberts, 1983;Roberts,
1988). And while contingent maternal attention to child compliance is commonly
5
used as a therapeutic tool in parent training packages, it is relatively ineffective
(as a stand alone treatment) in decreasing noncompliance (Forehand, 1986;Hobbs
et al., 1984;Roberts, 1985;Walle et al., 1984). These findings validate training in TO
procedures but cast doubt on the common practice of using differential reinforcement
of others (DRO) and physical punishment (e.g., spanking) as the sole means for
decreasing non-compliant behavior (Graziano & Diament, 1992). However, social
reinforcement (contingent and non-contingent attention) preceding TO has been
shown to enhance TO effectiveness (Hobbs et al., 1984;Walle et al., 1984), and may
increase the social validity of a parent training package.
2.3 Training Methods
A variety of presentation methods have been used to teach parents new skills in-
cluding: individual training, group training, didactic instruction, didactic instruc-
tion plus modeling, didactic instruction plus modeling and behavioral rehearsal,
audiotape instruction, and videotape. Early clinical work with PT utilized didac-
tic instruction, modeling and role playing (Hudson, 1982; Rickert et al., 1988) and
usually involved teaching one parent or one set of parents at a time. As technology
evolved, clinical researchers experimented with ways to deliver immediate coaching
and feedback to parents during practice sessions utilizing new technology such as
the ”bug in the ear” device (e.g., Eyberg, 1988).
As the need for PT grew, researchers began to explore other means of teach-
ing parents in search of the most efficient and effective method possible. Parent
Training groups and classes were formed and they relied heavily on verbal meth-
ods utilizing classroom style didactic lectures, written supplementary materials,
and group discussion (Webster-Stratton, 1996). Group versus individual training
procedures have been compared with no significant difference found in outcomes
(Brightman, Baker, Clark, & Ambrose, 1982; Pevsner, 1982; Webster-Stratton,
1984). Keating, Butz, Burke, and Heimberg (1983) found that training parents
and children at home, in the office, or parents alone at home resulted in equally ef-
fective treatment of children’s enuresis. Manuals and written advice packages have
been used to successfully teach parents to effectively manage specific problem be-
haviors. Giebenhain and O’Dell (1984) used a written manual and advice package
to teach parents how to reduce children’s bedtime fears. Children’s troublesome and
6
inappropriate behaviors in restaurants have been reduced through written ”advice
packages” for parents, and improvement generalized to a second restaurant (Bau-
man, Reiss, Rogers, & Bailey, 1983; Green, Hardison, & Greene, 1984). Perhaps as
a result of this early research, it is difficult today to find a child or family therapist
who provides in home parent training, and almost every therapist regardless of their
specialty uses some form of written material for clients.
While didactic methods have been found to be a cost efficient means of dis-
seminating information, they have been shown to be largely ineffective in inducing
behavioral changes in non-clinic referred parents (Webster-Stratton, 1981). In ad-
dition, these methods may be inappropriate for populations which could benefit the
most from PT - parents whose literacy, educational, occupational, or general intel-
lectual levels are deficient. By contrast, more active, performance based methods
have proven to be effective in producing behavioral changes in parents and children
(O’Dell, 1985). Baker and colleagues (Baker & McCurry, 1984; Prieto-Bayard &
Baker, 1986) evaluated a PT program that emphasized active rather than didactic
procedures and provided training in the parents’ native language with a population
of high risk (low SES, ESL) parents. They reported improvement in parental knowl-
edge and mixed improvement in child functioning at post-training. It is interesting
to note that in at least one study (O’Dell et al., 1982) demographic characteris-
tics and reading level were related to outcome not only for those parents receiving
didactic written instruction but also for those parents receiving live modeling and
rehearsal training. Based on the results of the study, O’Dell and colleagues sug-
gested that videotape training appeared to be more consistent in training a wider
range of parents. Hypothesizing that parents could enhance their parenting skills
by viewing and modeling videotape examples, Carolyn Webster-Stratton initiated a
program of research, in 1979, to develop and evaluate a videotape modeling program
for parents (Webster-Stratton, 1996).
Videotape modeling, it was believed, would be more conducive to learning
for parents less verbally facile, and that this training format would be less expensive
than individualized training. If such a program was found to be effective, it would
have the added advantage that it could be mass produced and widely disseminated
(Webster-Stratton, 1996). Webster-Stratton and colleagues developed their first in-
teractive videotape parent intervention program in 1980. The program was designed
for parents of children 3-8 years and consisted of viewing 10 videotapes totaling 26
7
hours (including 250 parenting vignettes) spread across 13-14 weekly two hour ses-
sions that included group discussions. The program has been shown to: improve
parental attitudes and parent-child interactions, reduce parental reliance on violent
discipline methods, and reduce child conduct problems (see Webster-Stratton, 1996,
for review). A later study (Webster-Stratton, Kolpacoff, & Hollinsworth, 1988) eval-
uated a totally self-administered condition using the videotape modeling program.
Parents had no contact with the therapist and no group support for the entire series
of treatment. Parents arrived for the session, were presented with the videotape
for that session, and were then asked to view the session in one of the clinic rooms.
Results showed that self-administered treatment resulted in significant improvement
in parental report of child conduct problems and observed parent-child interactions
(Webster-Stratton et al., 1988). While the individually administered treatment was
extremely cost effective, it was not as effective as the original program in terms of
consumer satisfaction and long-term outcomes (Webster-Stratton, Hollinsworth, &
Kolpacoff, 1989).
2.4 The Boys Town Model and CSP
”There are no bad boys. There is only bad environment, bad training, bad exam-
ple, bad thinking.” These are the words of Father Flanagan, the founder of Boys
Town, and they embody the spirit found in the various treatment services admin-
istered by Girls and Boys Town throughout their continuum of care. Boys Town -
and more specifically the Family Home program - is arguably the most famous and
effective residential treatment program for troubled youth in the world. The Boys
Town Family Home program (BTFH) is a descendant and adaptation of the Teach-
ing Family Model (TFM) created by Mont Wolf and colleagues at the University of
Kansas in the late 1960s (Handwerk, Field, & Friman, 2000; Kirigin & Wolf, 1998).
Both models rely on married couples to serve as resident Family Teachers. These
Family Teachers are trained in basic behavioral principles and interventions, and
they are responsible for implementing treatment plans for conduct problem children
and adolescents in a family home setting. The TFM consists of 4 principle hall-
mark components: a behavioral skills curriculum, a motivation system (i.e., token
economy), a self-government system, and frequent teaching interactions provided by
Family Teachers. The BTFH model adds two additional hallmarks: an emphasis on
8
moral and spiritual development, and an integration of physical and mental health
treatment components (Davis & Daly, 2003).
The CSP curriculum is adopted from the BTFH model and adapted for
use by parents in their own homes. The Common Sense Parenting program was
created to expose parents to elements of the BTFH model, and thus provide them
with tools to parent in a more respectful and effective manner. It stands to reason
that such a program might assist youth who have received treatment at Girls and
Boys Town in generalizing skills learned there to their natural home environment.
In fact, completion of the program by the youth’s custodial parent or parental
figure is required as part of the youth’s treatment. The program is presented in
a group format and the class meets for two hours once a week for a total of 6-
weeks. For those parents who cannot attend the CSP classes, it is requested that
they purchase and view the CSP Learn-at-Home Kit. The CSP program has been
shown to result in statistically significant improvements in reported child behavior
problems, parental attitude, and overall family satisfaction compared to a wait-list
control group (Ruma, Burke, & Thompson, 1996). To the author’s knowledge, the
effectiveness of the Learn-at-Home Kit has not been evaluated.
9
Chapter 3
Methods
3.1 Research Design
The first phase of the study involved collecting descriptive, qualitative data regard-
ing the number of referrals per source, number of contacts per source, demographic
information of contacts, and rates of attrition at each stage of the process includ-
ing descriptive data of variables impacting attrition. These attrition variables were
assessed using the Ecological Exit Interview (EEI). It was hypothesized that this in-
formation would be valued by outside referral agencies, and thus make it more likely
that they would allow the author to post recruitment materials in their public areas.
Due to the poor response to recruitment efforts at outside agencies, and to the low
overall response to the study, specific qualitative data regarding contacts per source,
rates of attrition, and variables affecting attrition were not collected. Demographic
information for families participating in the study was collected, however, and those
data are presented in the results.
The second phase of the study utilized a multiple baseline across subjects
design. Initial parent meeting times were coordinated across parents in order to
establish overlapping baseline phases and treatment start dates were staggered for
comparison purposes. Ideally, in such a design phase changes for individual parents
would be based on stability of the primary dependent variable (in this study, the
Parent Daily Report (PDR) data). However, due to the outpatient applied nature
of this research, demands on the parent’s schedule determined when they’d be able
to start the treatment phase of the study. A more complete description of the study
10
and the dependent measures follows.
3.2 Subjects and Setting
Subjects were parents of children age 3-7 years who reported that their child dis-
played difficult or disruptive behaviors and compliance problems. While neither the
recruitment materials nor the author specified which parent would participate in the
study, each parent-child dyad participating in the study consisted of a mother and
her child. Eight mothers and their children (6 boys, 2 girls) completed the study.
Pre-treatment and post-treatment assessment took place at the Girls and
Boys Town Outpatient Clinic in a standard therapy room. The room was equipped
with a desk and chairs for the parent to complete the written assessment measures.
A small video camera was set-up on a shelf in a corner of the room, and a variety of
toys were placed in the opposite corner. During videotaped parent-child interactions,
parents were asked to confine their play with their child to the front half of the room
in order to ensure interactions would be captured by the video camera. Daily data
collection occurred over the phone by calling the mother at home or at work during
a time that was convenient for her. During the treatment phase, the mothers viewed
the CSP videotape sessions in their own home.
3.3 Recruitment
Advertisement for the study initially consisted of stacks of numbered brochures
distributed around the community to various child-oriented agencies and public lo-
cations (e.g., pediatric dentist offices, public libraries, day care centers). Many of
these agencies refused to place the brochures in their public areas. In total, 6 dif-
ferent agencies agreed to place 20 numbered brochures in each of their public areas.
This initial recruitment strategy resulted in zero phone calls from interested parents
in the three weeks following the distribution of brochures. The second recruitment
strategy involved placing a classified ad within the Girls and Boys Town (GBT)
weekly electronic newsletter distributed to employees of GBT working in various
capacities at affiliate child oriented facilities spanning Girls and Boys Town’s contin-
uum of care (e.g., National Research hospital, pediatric clinic, residential treatment
center). In the week following the placement of the ad, researchers received calls
11
from 20 families interested in participating in the study.
3.4 General Procedure
3.4.1 Phase 1
When interested parents called to find out more information about the study, the
author provided a brief overview of the purpose of the study, collected contact infor-
mation from the parent, and obtained verbal consent to proceed with phase 1 data
collection. Once verbal consent was obtained, the author completed a brief demo-
graphic/family data interview with the parent over the phone. At the conclusion of
this initial interview, the author scheduled an initial meeting with the parent. The
entire telephone call took less than 10 minutes.
3.4.2 Phase 2
The initial meeting served as the ”kick-off session” for phase 2 of data collection.
During the initial meeting, the parent along with their child, met the author at
the Girls and Boys Town Outpatient Clinic. The author presented the rationale
for the study, explained the general time line and procedures involved, reviewed
the informed consent document, and obtained the first set of baseline measures.
These measures included: the Eyberg Child Behavior Inventory (ECBI; Eyberg &
Ross, 1978), the Parent Daily Report (PDR; Chamberlain & Reid, 1987), and the
Parenting Stress Index (PSI; Abidin, 1995). After these measures were complete,
the author instructed the parent on the procedure of the videotaped parent-child
interactions (V-PCI; using DPICS-R, Webster-Stratton, 1984). The videotaped
interactions included three separate 5-minute conditions (child game, parent game,
and clean-up). This first session lasted approximately 60 minutes. At the end of this
session, the author established a specific time to call the following day to complete
the PDR. Each subsequent PDR call ended with the author establishing a specific
call time for the following day.
The parent returned with their child approximately 3-7 days later to obtain
their CSP-Learn at Home Kit. This return date was initially to be based on the
stability of the PDR data. Due to the nature of conducting research in an applied
setting, however, the date was determined primarily by demands on the mother’s
12
schedule. During this second meeting, the parent completed the ECBI, PDR, and
the V-PCI. Following the completion of these measures the parent was given a copy
of the CSP-Learn at Home Kit from the author. The parent was instructed to view
at least two 30-minute video segments a week and no more than two per day. A
time for the next day’s PDR call was scheduled at the end of the session. The
author attempted to schedule a last treatment session if: the parent had viewed all
videotape sessions, the parent decided they no longer wanted to view the videotape
sessions, a period of two weeks passed with no videotape sessions being viewed, or
a total of three weeks had passed since the parent had received the CSP Learn at
Home Kit.
The parent and child attended the third and final treatment session together,
and the structure of this session was very similar to the first. The parent completed
the ECBI, PDR, and PSI. In addition, the parent completed a Consumer Satisfaction
Survey (CSS) and along with their child completed the final V-PCI. The author then
informed the parent that they would be receiving their gift card and consultation
certificate in the mail within the week. In addition, the parent was told they would
receive a follow-up phone call one month after the last treatment session, during
which the author would collect PDR data over the phone.
3.5 Independent Variable
The independent variable for this study was the Common Sense Parenting (CSP)
Learn at Home Kit. The CSP Learn at Home Kit consists of two videotapes (each
containing three 30 minute lessons) and a workbook to use in conjunction with the
lessons. The workbook is 172 pages long and consists of an introduction, six chap-
ters (each corresponding with a videotape lesson), two appendices, and an index.
Each chapter concludes with a section of frequently asked questions (and answers),
exercises to test knowledge acquisition, and homework assignments to put the skills
into practice. Subjects were considered to be ”in treatment” the day following
their second meeting when they received the CSP kit. During daily PDR calls in
the treatment phase, the author noted when video segments were viewed by the
mother. Table B.1 provides a brief content description for each video lesson.
13
3.6 Measures/Dependent Variables
The primary dependent variable for this study was the PDR. More specifically, the
Total Behavior score of the PDR. The primary target of the intervention was to re-
duce the amount of disruptive behavior exhibited by the child. Measures of parental
perception of the problem behavior, perception of life stress, and satisfaction with
the intervention were also administered.
3.6.1 Parent Daily Report (PDR)
The PDR consists of 33-items describing negative or disruptive behaviors commonly
exhibited by children. The PDR is administered in person or over the phone, with
the author asking ”In the last 24 hours has (child’s name) been” inserting the specific
problem behavior following the statement. For example, parents would be asked, ”In
the last 24 hours has Johnny been aggressive? In the last 24 hours has he argued?”
and so on. The parent gives a ”yes” or ”no” response. Total affirmative answers are
summed to yield a Total Behavior score. Items previously identified by the parent
as particularly troublesome are then summed to yield a Target Behavior score. The
PDR was administered daily by the author via phone calls made to each family.
Previous studies have reported the test-retest reliability of the PDR to range from
.62 to .82 (Chamberlain & Reid, 1987).
3.6.2 Eyberg Child Behavior Inventory (ECBI)
The ECBI is a 36-item inventory applicable for children 2-16 years, which measures
parental perceptions of their children’s behavior problems. Each of the 36-items
is assessed on two dimensions: the frequency of problem behavior occurrence and
identification of the behavior as a problem for the parent. Frequency ratings range
from (1) ”never occurs” to (7) ”always occurs”. Frequency ratings are summed
to yield an overall disruptive behavior Intensity Score. The problem identification
dimension asks the parent to rate whether each item is a problem for them by
circling ”yes” or ”no”. Affirmative answers are summed to yield a total Problem
Score. Previous research has obtained reliability coefficients for the ECBI scales of
.86 (test-retest) and .98 (internal consistency). The measure has also been shown
to differentiate non-clinic from clinic referred populations and to correlate well with
14
independent observations of children’s behaviors (Eyberg & Ross, 1978; Robinson,
Eyberg, & Ross, 1980).
3.6.3 Consumer Satisfaction Survey (CSS)
The CSS used in this study is a 6-item qualitative open response survey developed by
the author. Questions were based on common questions used to evaluate customer
perceptions of various products and services. The CSS asks parents to list the
tips and suggestions from the ”CSP-Learn at Home Kit” they found most helpful,
least helpful, their overall experience with the project, and tips and suggestions for
improvement. Parental responses are presented in the Appendix.
3.6.4 Family Data Questionnaire (FDQ)
The FDQ was developed by the author to serve as an initial screening instrument
and to gather information on the family home environment. This 12-item qualitative
measure provides information on: whether the target child meets inclusion criteria
for the study, the number of individuals living in the household (including their
age, occupation, and relation to the target child), the marital status of the parents,
estimated annual household income, education level of the parents in the household,
and a brief description of the problem behaviors exhibited by the target child.
3.6.5 Parenting Stress Index - Third Edition (PSI)
The PSI is a 120-item parent self-report measure designed to identify potentially dys-
functional parent-child relationships. The PSI is designed for the early identification
of parenting and family characteristics that fail to promote normal development and
functioning in children, of children with behavioral and emotional problems, and of
parents who are at-risk for dysfunctional parenting. Developed on the theory that
the total stress a parent experiences is a function of certain salient child charac-
teristics, parent characteristics, and situations that are directly related to the role
of being a parent the PSI yields a global Total Stress score, Child Domain score,
Parental Domain score, and Life Stress score. Past research has linked PSI scores
to observed parent and child behaviors and to child’s attachment style and social
skills (Abidin, 1995).
15
Chapter 4
Results
Subjects were run in matched cohorts based upon their first date of data collection.
Cohort 1 consisted of Subjects 03, 05, and 02. Cohort 2 included Subjects 01 and 06.
Subjects 08, 11, and 10 comprised Cohort 3. In order to facilitate comparison of data
between subjects and reflect the multiple baseline nature of the design, all figures
are grouped according to cohort. Table B.2 includes mean scores for PDR Total and
Target behaviors for both baseline and treatment phases. Results for ECBI and PSI
pre- and post-intervention are presented in Table B.3, B.4 and B.5 respectively. An
overview of the specific results obtained for each subject is presented below, a more
detailed discussion follows in the next section. Subjects are presented by cohort and
thus fall out of numerical order in terms of their assigned subject number.
4.1 Subject 03
Subject 03 is a 37-year-old married mother of three who works as a nurse. Her
husband is also 37-years-old and works as an architect. Their annual household
income is approximately $90,000. The highest level of education achieved by Mom
was a bachelor’s degree and Dad obtained his Master’s degree. Their children include
a 6-year-old son (the identified patient (IP)), a 4-year-old son, and an 18-month-old
daughter.
PDR data showed a sharp decelerating trend before intervention and a slight
decelerating trend post intervention. There was a noticeable decrease in the mean
number of total problem behaviors endorsed on the PDR during baseline and treat-
16
ment conditions, though the sharp decreasing trend during baseline coupled with
the wide variability in scores during intervention make it difficult to interpret this
difference. There is no clear change in level between the two phases. The frequency
of target behaviors remained relatively stable across both phases (Figure A.1). The
ECBI Intensity score at post-treatment (T3) was unchanged from baseline (T1)
level while the Problem score declined over the course of the study (Figure A.2).
Examination of the PSI data reveal that the subject’s level of Life Stress remained
unchanged after treatment. There was a slight decrease in child related stress while
there was a noticeable increase in total stress experienced due largely to the sharp
increase in parent and spouse related stress (Figure A.3).
Based on visual inspection of the data, one could conclude that the interven-
tion had a paradoxical effect slowing the rate of change in daily disruptive behavior
that was already occurring. Parental perception of disruptive behavior intensity
increased from treatment inception (T2) to treatment completion (T3). Perception
of intensity at the end of treatment was the same as the initial intensity perception
at the beginning of the study. The number of disruptive behaviors considered to
be problematic decreased steadily over time - providing little evidence of a positive
treatment effect functioning independent of time. Parental ratings of disruptive
behavior intensity and problems fell within the normal range at study entry (T1),
treatment onset (T2), and treatment completion (T3). Following intervention, the
level of reported parental stress increased substantially leading to a similar increase
in total stress. However, all scores still fell within the normal range. Treatment thus
does not appear to have been helpful, but it does not appear to have made matters
worse.
4.2 Subject 05
Subject 05 is a 35-year-old married mother of four who works as a medical assistant.
Her husband is a 41-year-old general laborer. Their annual household income is
$25,000-30,000. Mom has earned her associates degree and Dad earned his GED.
Their children include a 13-year-old son, a 10-year-old son, a 9-year-old son, and a
4-year-old daughter (the IP).
PDR data show a prominent downward trend before intervention and a slight
accelerating trend post intervention. There is a slight decrease in the mean number
17
of total problem behaviors endorsed on the PDR between baseline and treatment
conditions, though the degree of variability is about the same within both conditions.
There is no clear change in level between the two phases. The target behavior
scores were noticeably lower during the intervention phase, although equally low
scores were present during the last two days of baseline (Figure A.1). The ECBI
Intensity and Problem scores were highest immediately before treatment, but post-
treatment scores were almost equivalent to initial scores (Figure A.2). PSI domain
score analysis reveals a noticeable decrease in current Life Stressors, Total Stress,
and a significant decrease in the Child Domain scores. Parental Domain scores
remained about the same (Figure A.3).
Based on visual inspection of the data, one could conclude that treatment was
ineffective in changing daily disruptive behaviors exhibited by the youth. Parental
perception of disruptive behavior intensity stayed relatively constant from entry
into the study through treatment completion. The number of behaviors viewed as
a problem by the parent was relatively unchanged from study entry to treatment
completion. The parents total reported stress fell to within normal limits after treat-
ment, with a significant decrease in stress related to child characteristics. However,
this change coincided with a decrease in life stressors making it difficult to determine
if positive changes in stress level were due to treatment specific variables or other
ecological variables. Based on the convergence of data, treatment would appear to
have been ineffective for this subject.
4.3 Subject 02
Subject 02 is a 26-year-old single mother of one who works as an office assistant.
Her annual household income is approximately $17,000. She has earned her high
school diploma and is taking night classes at a local college. Her son is 3-years-old.
PDR data show a wide range a variability within both baseline and treat-
ment conditions. There is no clear difference in the mean number of total problem
behaviors endorsed between baseline and treatment conditions. There is a very
slight accelerating trend during baseline and a slightly more noticeable downward
trend during treatment. There is no clear change in level between phases, and target
behavior scores remained consistent across both conditions (Figure A.1). The ECBI
Intensity and Problem scores were highest immediately prior to treatment and re-
18
mained above the clinical cut-off level at post-treatment (Figure A.2). PSI data
analysis suggests that the subject’s level of stress - already extremely high during
baseline - increased across all four domains between T1 and T3 (Figure A.3).
Based on visual inspection of the data, treatment appears to have been in-
effective in decreasing the number of daily disruptive behaviors exhibited by the
youth. Parental perception of the intensity of disruptive behaviors and of the num-
ber of disruptive behaviors judged to be a problem was identical at study entry and
the beginning of treatment. Very slight improvement can be seen at treatment ter-
mination, although both the intensity and problem score remained in the clinically
distressed range. Parental stress increased and remained in the clinically distressed
range following treatment. Convergence of the data suggests that treatment was
ineffective and may have had iatrogenic effects.
4.4 Subject 01
Subject 01 is a 22-year-old married mother of two who works as an office assistant.
Her husband is 24-years-old and works as a general laborer. Their annual household
income is approximately $22,000. Mom has earned her high school diploma and
Dad has completed some college courses. Their children include a 5-year-old son
(the IP) and a 2-year-old son.
PDR data show an accelerating trend during baseline and a slight decelerat-
ing trend during treatment. There is no clear difference in the mean number of total
problem behaviors endorsed between baseline and treatment conditions. There is a
high degree of variability for total behaviors endorsed within both the baseline and
treatment conditions with no clear difference in level between the phases. There is a
slight decelerating trend for target behaviors during baseline and that trend contin-
ued during treatment (Figure A.4). The ECBI problem score was highest just prior
to treatment, and both the Intensity and Problem score remained above the critical
cut-off level at post-treatment (Figure A.5). PSI data analysis reveal that the sub-
ject’s current Life Stressors increased between pre- and post-treatment while Total
Stress decreased largely due to a significant decrease in the Parent Domain score.
The Child Domain score remained about the same between pre- and post-treatment
(Figure A.6).
Based on visual inspection of the data, one could conclude that the inter-
19
vention had very little to no effect in reducing the amount of disruptive behavior
exhibited by the IP on a daily basis. There was little to no change in parental percep-
tion of the intensity and problematic nature of the child’s behavior from study entry
to termination. Parental ratings of the child’s disruptive behavior still fell within
the clinically distressed range following treatment. Following intervention, the level
of reported parental stress fell from clinically distressed levels to well within the
normal range helping to lower the parent’s total stress level to just below critically
distressed levels. While treatment does not seem to have effectively lowered daily
disruptive behavior, it may have helped lower parental stress in spite of the fact the
subject reported additional life stressors at treatment termination. Convergence of
the data would suggest that treatment was minimally effective.
4.5 Subject 06
Subject 06 is a 40-year-old married mother of three who works as a department
manager. Her husband is also 40-years-old and works as a construction manager.
Their annual household income is $80,000. Both parents have earned their bachelor’s
degree. Their children include a 7-year-old son, a 4-year-old son (the IP), and an
11-month-old daughter.
PDR data inspection reveals a decelerating trend during baseline that con-
tinues during the treatment condition. There is a noticeable difference in the mean
number of total problem behaviors endorsed during baseline and treatment con-
ditions. There is a high degree of variability during both baseline and treatment
conditions, with no clear change in level between the two conditions. There is
a noticeable difference in target behavior means between baseline and treatment
conditions (Figure A.4). ECBI Intensity and problem scores were highest during
baseline assessment and lowest at post-treatment. However, a downward trend in
Problem scores is evident prior to treatment (Figure A.5). PSI data inspection
shows all four domains below the clinical range at pre-test and a noticeable decrease
in three of the four domains at post-test (current Life Stressors remained about the
same) (Figure A.6).
Based on visual inspection of the data, one could conclude that the interven-
tion had a slight effect in reducing the amount of disruptive behavior exhibited by
the IP on a daily basis. Or, at least the intervention did not hamper positive changes
20
that were already taking place. Parent perception of the intensity and problematic
nature of the child’s behavior decreased from study entry to treatment termination,
with parental ratings of the child’s behavior moving further into the normative range
following treatment. Following intervention, the level of reported parental stress fell
further within the normal range with the concomitant effect of lowering the parent’s
total stress level to well within the normal range. Convergence of the data suggests
that treatment was slightly effective - helping to reduce daily disruptive behaviors
displayed by the youth as well as reducing parental stress.
4.6 Subject 08
Subject 08 is a 36-year-old married mother of two who works as a research analyst.
Her husband is also 36-years-old and works as a credit analyst. Their annual house-
hold income is $70,000. Both parents have earned master’s degree. Their children
include a 9-year-old daughter and a 4.5-year-old son (the IP).
PDR data inspection reveals a very slight decelerating trend during baseline,
while a slight accelerating trend is evident during the treatment phase. The mean for
total problem behaviors during treatment is slightly greater than the mean during
baseline. Variability in total problem behaviors is consistent across phases and
there is no clear change in level from baseline to treatment conditions (Figure A.7).
ECBI Intensity scores were identical at T1 and T3. ECBI Problem scores were
identical across all three time periods (Figure A.8). PSI data inspection shows all
four domains below the clinical range at pre-test and post-test, with no change
observed in the Life Stress and Child Domains. There was a slight increase in the
Parent Domain score leading to a slight increase in the overall Total Stress score
(Figure A.9).
Based on visual inspection of the data, one could conclude that the interven-
tion was ineffective in reducing the amount of daily disruptive behavior exhibited
by the youth. Parental perception of the intensity of disruptive behaviors and of
the number of problematic behaviors were unchanged and within normal limits at
study entry, treatment onset, and treatment termination. Parental stress was very
low and well within normal limits before and after treatment. Convergence of the
data suggests that treatment was ineffective across all dependent measures.
21
4.7 Subject 11
Subject 11 is a 36-year-old married mother of three who works as an auditor. Her
husband is a 33-year-old independent business owner. Their annual household in-
come is $65,000. Both parents are college educated with Mom holding an MBA and
Dad earning a Bachelor’s degree. Their children include a 6-year-old daughter, a
3-year-old son (the IP), and a 5-month old daughter.
PDR data inspection reveals a clear decelerating trend during baseline, and
a very slight decelerating trend is evident during the treatment phase. The base-
line mean for total problem behaviors is slightly greater than the treatment mean.
Greater variability in total problem behaviors is evident during the treatment phase
compared to baseline. There is a slight increase in level from baseline to treatment
conditions (Figure A.7). ECBI Intensity score increased between T1 and T2 and
fell to its lowest level at T3. ECBI Problem score increased from T1 to T2 then fell
at T3 with the T1 score being the lowest overall (Figure A.8). PSI data inspection
shows all four domains below the clinical range at pre-test and post-test, with no
change observed in Life Stress at treatment termination (Figure A.9).
Based on visual inspection of the data, one could conclude that the interven-
tion was ineffective in reducing the amount of daily disruptive behavior exhibited
by the youth. Parental perception of the intensity of disruptive behaviors and of
the number of problematic behaviors increased from study entry to treatment on-
set with both scores falling in the clinically distressed range at treatment onset.
Both scores fell to just below the clinically distressed range at treatment comple-
tion, although scores at treatment completion were very similar to scores at study
entry. Total parental stress increased after treatment, with a substantial increase
in stress related to child traits and characteristics. Stress specific to spousal and
parental roles decreased slightly after treatment. Convergence of the data suggests
that treatment was ineffective but not harmful.
4.8 Subject 10
Subject 10 is a 27-year-old married mother of three who works as a patient represen-
tative. Her husband is a 28-year-old truck driver. Their annual household income is
$35,000-40,000. Both parents have completed some college class work. Their chil-
22
dren include a 7-year-old daughter (the IP), a 4-year-old daughter, and a 2-year-old
daughter.
PDR data inspection reveals a clear accelerating trend during baseline, and
a clear decelerating trend during the treatment phase. The baseline mean for total
problem behaviors is clearly greater than the treatment mean. Greater variability in
total problem behaviors is evident during the treatment phase compared to baseline.
There is a clear change in level from baseline to treatment conditions (Figure A.7).
ECBI Intensity scores decreased steadily over T1, T2, and T3. ECBI Problem
score also fell steadily from T1 to T3 (Figure A.8). PSI data inspection shows all
four domains well above the clinical range at pre-test and post-test, with very little
change (Figure A.9).
Based on visual inspection of the data, one could conclude that the inter-
vention was effective in reducing the amount of daily disruptive behavior exhibited
by the youth. Parental perception of the intensity of disruptive behaviors and of
the number of problematic behaviors decreased steadily from study entry through
treatment onset and treatment termination, though scores continued to fall within
the clinically distressed range after treatment completion. Total parental stress and
stress related specifically to parental responsibilities and roles remained about the
same from study entry to treatment completion. Convergence of the data suggests
that treatment was effective in reducing the amount of disruptive behavior displayed
by the youth, slightly effective in altering parent perception of the problem behavior,
and ineffective in reducing parental stress.
4.9 Visual Inspection Summary
Seven out of eight subjects showed a decrease in mean PDR Total Behavior scores
from baseline to treatment. Only one subject (Subject 10) showed a clear change in
terms of mean, level, and trend. Subject 6 showed a change in level and mean, but
the trend stayed the same across phases. Subject 3 showed a change in mean, but
not level or trend. Subject 1 showed a clear change in trend, but not level or mean.
Thus, via visual inspection one could conclude that Subject 10 clearly improved.
The intervention did not hamper the improvements already underway for Subjects
6 and 3 - although it is questionable whether treatment provided any additional
benefit. A weak case could be made that intervention kept Subject 1 from getting
23
worse, but for most subjects the intervention had no noticeable effect on the daily
occurrence of child behavior problems.
It is possible, however, that clinically relevant changes occurred across the
group that were not captured by the PDR ratings. Perhaps the intervention proved
effective in alleviating concerns in other domains. If the intervention was unsuc-
cessful in altering the ”doing” (i.e., child behaviors) of the problem, then perhaps
it was effective in altering the ”viewing” (i.e., parental perception) of the problem.
Or, perhaps the intervention was successful in addressing a set of more macro level,
ecological variables such as those falling under the conceptual umbrella of ”parental
stress”.
Quick visual inspection of the ECBI intensity score data shows no substantial
improvement (in terms of degree of change) for subjects 03, 05, 02, 01, and 08. A
slight improvement is noticeable for subject 10 and clear improvements are evident
for subjects 06 and 11. In terms of categorical improvement, three subjects that
began the study in the clinically distressed range on this scale also ended the study
in the clinically distressed range. Four subjects began in the normal range and
finished in the normal range. One subject began the study in the distressed range
and ended in the normal range, and none of the subjects began in the normal range
only to wind up in the distressed range at the conclusion of the study.
Visual inspection of the PSI Total Stress and Parental Domain scores reveal
no substantial improvement (in terms of degree of change) for subjects 03, 02, or
11. Subject 5 had a noticeable reduction of stress at treatment termination. Slight
improvements in stress level were visible for subjects 01 and 06, while increased
levels of stress were noted for subjects 03, 02, and 11. There was no change in
parental stress for subjects 8 and 10. Categorically, two subjects began the study
in the clinically distressed range of the Total Stress scale and fell to within normal
limits at termination. Two subjects began the study in the clinically distressed
range and still fell within that range after treatment. Four subjects began the study
in the normal range and remained there at treatment termination.
4.10 Clinical Significance
While it is interesting to investigate whether an intervention produces any change
in behavior, it is much more relevant and powerful clinically to ask whether any
24
observed change is actually helpful to the client. Clinicians should concern them-
selves not only with answering the question ”Was change produced?” but also with
answering ”Was the change produced important or clinically significant?” Clinical
significance refers to a level of change following treatment that is both meaningful
and reliable. Change is meaningful if a once troubled and clinically distressed indi-
vidual is, after treatment, indistinguishable (along the dimension of interest) from
representative others who are not distressed (Kendall & Grove, 1988). In other
words, the individual is returned to a state of functioning that is within the nor-
mal range. Change following treatment is reliable if the level of change is unlikely
explained by measurement error alone (Jacobson & Truax, 1991; Ogles, Lambert,
& Masters, 1996). That is, the change is of sufficient magnitude that it cannot be
accounted for merely by the variability inherent in the psychological measure used
to evaluate treatment outcome.
Visual inspection of data is adequate for identifying powerful treatment
strategies with strong treatment effects - the type of interventions highly valued
by all. Visual inspection is also valuable in making categorical judgments regarding
treatment effectiveness (i.e., moving from above a clinical cut-off score to below that
score). Visual inspection is also less likely to lead to what statisticians refer to as
Type I errors - that is, concluding the intervention had an effect when in reality it
did not. Unfortunately, visual inspection may not be sensitive enough to detect sub-
tle (but none the less real) treatment effects increasing the chance of making more
Type II errors - concluding the intervention had no effect when in fact it did. Thus
a clinician relying solely on visual inspection may prematurely abandon an inter-
vention that holds some potential promise. Fortunately, psychotherapy researchers
have developed a variety of tools to assist in measuring clinical significance. The
following section illustrates the clinical utility of one of these tools - the Reliable
Change Index.
4.10.1 Reliable Change Index
The Reliable Change Index (RCI) is defined as the difference between a post-test
score and a pretest score divided by the standard error of measurement (SEmeas).
The standard error of measurement accounts for the variation or spread of change
scores that could be expected if no actual change had occurred. The standard
25
error of measurement for the following analyses was calculated using the following
formula: SEmeas = SD ×√
1 − r where SD is the standard deviation from the
normative sample for the measure and r is the reliability coefficient for the measure.
Traditionally, Cronbach’s alpha is used for the r value although alternatively one
could use the test-retest reliability coefficient. By convention, an RCI of 1.96 (p <
.05) would be considered to be reliable. An RCI greater than 1.96 indicates the
difference is reliable. In other words, a change of that magnitude on this particular
instrument is not likely due to measurement error alone. Conversely, RCI scores of
1.96 or less would not be indicative of reliable change as changes of this magnitude
could be due solely to the unreliability of the measure. The 1.96 value reflects a
general consensus that ”normal” can be defined as scores falling within one standard
deviation (SD) of the mean of the non-distressed or normative reference group for
that measure. By multiplying 1.96 by the SD of an outcome measure, we can define
a 95% confidence interval which may serve as an aid when interpreting results.
RCI scores for the Total Behavior scale of the PDR were calculated based
on descriptive statistics of the normal sample (x = 5.33, SD = 3.10, r = 0.82,
SEmeas = 1.32). The SEmeas for the PDR Total Behavior scale was calculated
using the test-retest reliability coefficient as the author was most interested in the
normative changes that could be expected to occur over time. Based on the findings
of Chamberlain and Reid (1987) (1987) suggesting that PDR Total Behavior scores
are inflated on Day 1 but highly stable after that, two RCI scores were calculated for
each subject. The first (RCI) was calculated using the unadjusted baseline mean.
The second (RCIa) was calculated after dropping the first baseline data point for
Total Behavior score. RCI scores ranged from 1.29 to -3.56, and RCIa scores ranged
from 2.11 to -3.42 (Table B.6). RCI scores suggest that two subjects showed reliable
improvement following treatment. RCIa scores reveal two subjects reliably improved
while one reliably deteriorated following treatment.
The same information is presented in a slightly different way in Figure A.10.
Here every dot corresponds with an actual subject. Dotted lines represent a 95%
confidence interval (calculated by multiplying 1.96 by the SD) surrounding a solid
diagonal line of no change. The solid horizontal line represents 1 SD above the
mean score for the normative sample. Thus, dots falling to the left of the upper
dotted line represent subjects who got reliably worse following treatment. Those
falling to the right of the lower dotted line but above the horizontal line represent
26
subjects that showed reliable improvement following treatment but did not return
to within normal limits. Those to the right of the lower dotted line and below the
horizontal line showed reliable improvement that returned them to a normal level
of functioning. Those dots falling between the dotted lines represent unreliable or
uncertain change. This type of graphical display of the data may be more clinically
useful than the RCI values themselves in that a quick glance allows one to see
whether reliable change was also clinically relevant in terms of moving the subject
from a distressed level of functioning to a non-distressed, normal level of functioning.
Looking at Figure A.10 one can clearly see that of the two subjects who showed
reliable improvement based on RCI scores, one returned to a level of functioning
within the normal range. A quick glance at the RCIa figure shows the previously
mentioned improved subjects and also reveals the subject who deteriorated following
treatment.
RCI scores for the Intensity scale (x = 96.6, SD = 35.2, r = 0.86, SEmeas =
13.17) and Problem scale (x = 7.1, SD = 7.7, r = 0.88, SEmeas = 2.67) of the
ECBI as well as the Total Stress (x = 222.8, SD = 36.6, r = 0.95, SEmeas = 8.184)
and Parental Domain (x = 123.10, SD = 24.4, r = 0.93, SEmeas = 6.456) scale
scores for the PSI were also calculated. The SEmeas for the ECBI was calculated us-
ing test-retest reliability coefficients as once again the author was mostly concerned
with normative change over time. The SEmeas for the PSI on the other hand was
calculated using Cronbach’s alpha, as the measure of internal consistency seems to
be most apropos when evaluating normative change over time for a heterogeneous
construct such as parental ”stress”. RCI scores for the ECBI Intensity scale range
from -3.95 to 2.658 and -3.37 to 2.247 for the Problem scale. The range in RCI scores
for the PSI Total Stress domain is -2.69 to 5.25 and the range for the Parent Domain
is -3.10 to 4.49. RCI results for the ECBI and PSI are presented in Table B.7 and
Table B.8 respectively. Figure A.11 presents data on ECBI changes from T1 to T2,
Figure A.12 presents changes from T2 to T3, and Figure A.13 presents changes from
T1 to T3. PSI changes are visible in Figure A.14.
The RCI results for the ECBI Intensity Scale reveal that one subject was
functioning reliably worse prior to intervention when compared to the same sub-
ject’s level of functioning at study entry. Changes for all other subjects from study
entry to treatment onset fell within the unreliable range. A similar pattern is seen
with the ECBI Problem Scale data for study entry to treatment onset. One subject
27
demonstrated significant clinical improvement on the ECBI Intensity scale from
treatment onset to treatment completion, while the change for other subjects re-
mained within the unreliable band. For the same time period (treatment onset to
treatment completion), three subjects showed borderline improvement on the ECBI
Problem Scale although the change was still unreliable. Only one subject showed
reliable (though not clinically significant) improvement on the ECBI Intensity scale
from study entry to treatment completion. Two subjects showed reliable change of
clinical importance on the ECBI Problem scale from study entry to treatment com-
pletion. Three subjects showed reliable improvement in Total Stress scores of the
PSI from study entry to treatment completion, with two of those subjects falling
within the normal range at treatment completion while the other still fell within
the clinically distressed range. Three subjects showed reliable deterioration in Total
Stress from study entry to treatment completion with one of the three falling within
the clinically distressed range. Only one subject demonstrated reliable improvement
in Parental Domain stress levels, while three subjects demonstrated reliable deteri-
oration in Parental Domain stress levels (with one of those falling in the clinically
distressed range).
4.11 Social Validity
Finally, clinicians and therapists - especially those in private practice - must concern
themselves with an additional outcome variable. Not only should they evaluate an
intervention based on the behavioral and performance outcome desired, but they
must also evaluate the intervention based on its social validity. Clinicians must
concern themselves with whether or not the client felt comfortable with the inter-
vention prescribed. In a competitive market, the therapist must win the hearts and
minds of his clients. It may not be enough to produce results. It is likely that the
results would need to occur in a context where the client understands why certain
actions are being prescribed, accepts the level of effort required by the intervention,
and views the results as satisfactory given the time, energy, and effort invested in
achieving those results.
Consumer satisfaction surveys (CSS) are a quick and dirty means of assessing
the social validity of services rendered. The CSS used in this study consisted of the
following six questions. What suggestions were implemented in the home? What
28
suggestions were most helpful? What suggestions were least helpful? What changes
or suggestions for improvement would you have based on your experience with the
intervention? Would you recommend the intervention to family and friends who were
facing similar challenges? On a scale of 1 (very unsatisfied) to 10 (very satisfied), how
would you rate your experience in this study? Overall, social validity for the CSP
Learn-at-Home Kit was very high. Every mother reported she would recommend
the program to family and friends, and 6 of 8 mothers rated their satisfaction as an
8 or higher.
29
Chapter 5
Discussion
This study, a simple multiple baseline across subjects investigation, was designed
to evaluate the effectiveness of the Common Sense Parenting (CSP) Learn-at-Home
Kit as a clinical intervention. The CSP Learn-at-Home Kit was chosen as an exam-
ple of the many types of self-help parenting videos available for those parents whose
children display disruptive and non-compliant behaviors (e.g., Webster-Stratton,
1992; Sanders, 1996; Clark, 1991; Phelan, 2001). The results of the study indicate
that the CSP Learn-at-Home Kit was largely ineffective in changing child disruptive
behaviors. Furthermore, the intervention was ineffective in altering parental per-
ceptions of the problem behavior, and had indeterminate effects on parental stress
levels. Some possible explanations for the ineffectiveness of the CSP Learn-at-Home
Kit are presented in the following sections.
5.1 More Than Common Sense
Pat Friman is fond of saying that all children (and indeed all humans) learn through
an active process of repetition with experiential contrast. We emit behaviors, and
we are exposed to events, that have an effect on the world around us. The larger
the experienced effect the greater the contrast and the fewer repetitions required
to learn (i.e., fewer trials are needed to associate the behavior or event with the
corresponding effect). Using these broad categories as our vantage point, we can
take a closer look at some potential reasons why the CSP Learn-at-Home Kit might
be ineffective in teaching parents how to alter their children’s behavior.
30
First, let’s consider the issue of repetition. The Webster-Stratton videotape
modeling program consists of 26 hours of footage (including 250 parenting vignettes)
demonstrating the proper and improper use of various parenting techniques. The
CSP Learn-at-Home Kit on the other hand consists of a mere 3 hours of footage
with less than 30 parenting vignettes demonstrating skills traditionally covered in
the CSP training class. The difference between the sheer number of vignettes il-
lustrating examples and non-examples is staggering. Thirty trials may be all the
repetition that is necessary to establish a rather simple and novel behavioral re-
sponse for a healthy individual under ideal training conditions; however, it is likely
to be far too few to teach new complicated behaviors intended to compete with well
established functional (albeit maladaptive) behaviors to an individual who is psy-
chologically distraught and trying to learn in less than ideal conditions. In addition,
a large body of research exists that demonstrates the way teaching examples are
presented affects the quality of learning that occurs (see Engelmann & Carnine,
1991). Learning is more likely to take place when examples are given at a high rate,
juxtaposing examples and non-examples so that examples span the wide range of
possible correct responses and non-examples differ minimally from examples to fine
tune discrimination based on critical controlling variables. The CSP Learn-at-Home
Kit likely provides too few examples at too slow a rate for reliable fine discrimination
training to occur.
Repetition (frequency and rate) is important in discrimination training, but
this training has relatively little value unless the new discriminative stimuli are likely
to evoke new adaptive responses. Here again, repetition plays a critical factor. The
Learn-at-Home Kit may not provide enough opportunities for parents to practice
the skills being taught. More correctly stated, the Learn-at-Home Kit may not (in-
deed cannot) produce the social contingencies that would make parent rehearsal of
new adaptive responses more likely. The tapes cannot provide immediate reinforce-
ment for emitting the desired behaviors, nor can they systematically shape closer
approximations to the desired parental behaviors. The Common Sense Parenting
class may provide these necessary social contingencies, but these social contingencies
cannot be packaged with the videotapes. Even when necessary social contingencies
are constructed (as in the CSP class), new adaptive responses may not replace mal-
adaptive parenting behaviors unless the adaptive behaviors are trained to a fluent
level and/or they are practiced repeatedly under the same internal and external
31
conditions in which they are likely to be needed in the future.
Let’s now turn our attention to the idea of contrast. Let’s assume that the
parents watching the tape are really invested in becoming better parents. They
watch the tape with great interest and encounter tips and techniques that they have
never in their whole life considered using, and they encounter slight variations on
techniques they have attempted in the past with little success. The ideas that are
likely to seem powerful are those with the greatest contrast (i.e., those that the
parent has never tried before), and by extension those ideas with little contrast
(i.e., those the parent has tried before) will seem less powerful. The parent is likely
to invest a substantial amount of time, energy, and effort in applying those new
techniques felt to be most powerful. In essence, a selection bias of sorts may be
operating affecting the skills a parent chooses to practice and apply. This bias may
contribute to poor outcomes.
Many parents have heard a chorus of benevolent concerned others tell them
to pay more attention to their children, spend more time with their children, and
use time out with their children. These suggestions then are unlikely to show up
as new or powerful for the parent. On the other hand, few of these same people
have discussed the importance of ”preventive teaching” or ”corrective teaching” let
alone provided parents with a script for these teaching interactions. Given this stark
contrast, it is likely that many parents will see the problem as not providing enough
verbal teaching interactions during discipline encounters. So, they adjust to do more
talking and provide more reasons and rationales during discipline situations. Unfor-
tunately, the strategy of talking more during discipline interactions may strengthen
attention maintained or escape maintained disruptive behaviors. There is nothing
wrong, per se, with providing these scripts, it is just unlikely that parents will learn
to apply them prescriptively with so few teaching examples to help the parent dis-
cern when it is appropriate to use the script and when it is not given the function
of their child’s behavior. In fact, relatively little attention is paid to the function of
child disruptive behavior at all within the CSP program. The prevailing theme is
not that children do what works for them, but rather that children do what they do
because they don’t know any better. The emphasis on teaching may lead to more
talk and less action (in terms of applying consequences) from the parents, which
may contribute to poor outcomes.
There is one final way in which this idea of contrast can help us understand
32
the relative ineffectiveness of the CSP Learn-at-Home Kit in modifying child behav-
iors in this study. The contrast created by juxtaposing a rich time-in condition and
a stark time-out condition is perhaps the single most powerful tool in addressing
child disruptive and non-compliant behavior, yet coverage of time-out procedures
in the CSP Learn-at-Home Kit is limited to 1 page in the workbook, and 1 minute
of videotape time. The Learn-at-Home Kit devotes a lot of time and space to the
discussion of natural consequences for disruptive behaviors (e.g., withholding privi-
leges and adding extra chores); however, it does not appear to stress the contingent
nature between behavior and the immediate response from the environment. In ad-
dition, it does not spend a lot of time discussing the setting events and establishing
operations that may influence the power of consequences. Failing to account for
these variables may contribute to poor outcomes.
It appears that the Common Sense Parenting Learn-at-Home Kit focuses
primarily on those variables that may enhance treatment acceptability and increase
social validity. In doing so, however, it may not provide enough education for parents
in the areas of behavioral contingencies and the relationship between contextual
variables, child behaviors, and the environment’s response to be an effective clinical
intervention. But that’s not to say the CSP Learn-at-Home Kit is not valuable. The
value of the Learn-at-Home Kit may be heavily influenced by the broader systemic
context of its use. For parents with children receiving services within the Girls and
Boys Town continuum of care, the CSP Learn-at-Home Kit may be effective as a
universal or selected (to borrow terms from the prevention literature) intervention.
Outside of that system, outside of that specific social context, the therapeutic value
of the CSP kit may be questionable.
Understanding the role of context is essential in understanding and appre-
ciating effective parent training. Therapists often address six broad areas when
providing parent training in real-world clinic settings. These areas are: parental
monitoring of child activities, catching kids being good, giving effective commands,
using non-coercive discipline strategies, increasing emotional intelligence, and de-
veloping effective problem solving strategies. However, therapists who are very
successful in working with parents do not address all of these areas in a predeter-
mined sequence with each client that walks through their door. They tailor their
interventions to the unique circumstances of each family situation. For example, one
set of parents may have unrealistic expectations of their child and their complaints
33
of child noncompliance may in fact reflect child behavior that is developmentally
appropriate. In this case, redirecting the parents to observe the child closely and fo-
cus on catching the child being good combined with subtle psychoeducation about
child development may be all that is necessary to bring about important clinical
change. On the other hand, a single parent whose primary form of discipline for
his 6-year old strong willed son consists of a pattern of reasoning, raising the voice,
and raising his hand (for light swats on the bottom) may need an intervention
addressing increasing the son’s frustration tolerance through non-coercive discipline
combined with emotional coaching provided by Dad. In this way, Dad is encouraged
to change his behavior towards his son not because he is a bad parent but because
his son needs help developing important life skills. These type of subtle but powerful
therapeutic maneuvers are necessarily prescriptive and cannot be mass produced or
disseminated.
There are a number of limitations with this study. One major limitation of
this study is that - due to the nature of the design - conclusions regarding the effi-
cacy of the intervention cannot be generalized outside the study. Second, there was
a restricted range of subjects in this study (i.e., mothers of primarily young male
children). Third, the author cannot be certain that parents watched the complete
videotape segments when they reported that they watched them, nor can he know
for sure whether or not the related pages of the workbook were completed while
viewing the tapes. Finally, it is possible that subtle changes in parent child interac-
tions occurred during the course of the study, but went unnoticed due to difficulties
with the coding system. Future analysis of videotaped interactions are planned to
determine the level of relationship change displayed by the mother-child dyad.
While there are significant limitations to the current study, it can serve as a
catalyst for future research and discussion. Small scale effectiveness studies could
be conducted to evaluate changes in the videotape presentation based on the vari-
ables discussed previously (e.g., frequency, rate, diversity of clinical vignettes). In
addition, larger group studies could compare families involved with GBT services
to families distressed but not involved in GBT services to explore whether larger
systemic variables influence the efficacy of the CSP Learn-at-Home Kit. If we ac-
knowledge the influence of broader contextual variables on the effectiveness of an
intervention, then we must also acknowledge the influence of broader systemic vari-
ables on our own behavior as scientists and practitioners. The systemic influences
34
lurking in the background of this study, and thus deserving of some comment, in-
clude: distinguishing between efficacy and effectiveness research, understanding the
gap between psychotherapy research and psychotherapy practice, and exploring the
role of trained Scientist-Practitioners in advancing clinical science and practice. The
remainder of this paper is devoted to the discussion of these distinctions.
5.2 Efficacy vs. Effectiveness
During the last decade of the 20th century, psychotherapy researchers began to dis-
tinguish between two different approaches to psychotherapy research - the efficacy
model and the effectiveness model (Nathan & Gorman, 2002). Largely, this distinc-
tion resulted from an attempt to understand the low value most psychotherapists
and other mental health practitioners placed on psychotherapy research. Histor-
ically, practitioners were hesitant to utilize psychotherapy research outcomes to
inform their practice because of inadequate research methods and relatively small
treatment effects. Research methods continued to improve and treatment effects
continued to grow more robust. By the late 1980s and early 1990s, the prevailing
concern shifted to a lack of correspondence between the interventions and subjects
chosen for empirical study, and the treatments and clients seen in a real world clin-
ical practice. Said differently, while practicing clinicians acknowledged the efficacy
of the treatments they questioned the effectiveness of those same treatments.
Outcomes of efficacy studies have been characterized by Barlow (1996) as
”the results of a systematic evaluation of the intervention in a controlled clinical re-
search context. Considerations relevant to the internal validity of these conclusions
are usually highlighted” (p. 1051). Efficacy studies emphasize internal validity and
replicability, and in so doing - many clinicians feel they have little external validity.
In contrast, Barlow described effectiveness studies as primarily concerned with ”the
applicability and feasibility of the intervention in the local setting where the treat-
ment is delivered” and designed to ”determine the generalizability of an intervention
with established efficacy” (p. 1055). In effectiveness studies, clinical considerations
rather than the strict demands of the research design generally dictate the choice of
treatment, as well as the frequency, duration, and means of assessing outcome. For
this reason, researchers argue that they lack acceptable controls to preserve internal
validity. Neither effectiveness studies in isolation, nor efficacy studies alone provide
35
the clinician with all the information necessary to adequately inform clinical judg-
ment at every stage of the treatment process for a particular client. However, the
two processes are theoretically complimentary and new approaches are being devel-
oped that will take advantage of the strengths of both to provide more clinically
relevant information to psychotherapists.
The efficacy-effectiveness distinction was introduced (at least in part) as an
explanation for the gap that exists between clinical practice and clinical research.
And while the distinction may be useful in understanding the world views that
contribute to the divide, the author invites the reader to consider that there is
nothing inherently wrong with the gap. There is nothing in need of fixing. Indeed,
the gap may function systemically not as a barrier to progress but as the engine of
change driving the evolution of both domains. Perhaps what is needed is not more
vigorous attempts to converge but an acknowledgment and acceptance of the gap
”as is” and an appreciation for the value of work pursued by psychologist living and
working on either side.
5.3 Clinical Research versus Clinical Practice
As previously mentioned, much has been made about the perceived divide between
clinical research and clinical practice. Clinical researchers have lamented the the low
value therapist in general continue to place on psychotherapy research. Psychother-
apist routinely complain that there is little value in much clinical research because
it lacks external validity and real world application. Both groups are convinced that
they are right in their perception, and perhaps they are. Perhaps the problem is
that both groups speak of psychology and mistakenly think they are speaking about
the same thing. To illustrate this example lets imagine a similar impassioned divide
among another group of professionals - say fishermen.
Now for our purposes we will grossly simplify things and recognize that all
of the world’s professional fishermen can be roughly divided into two camps - fish-
ing boat anglers and independent fishing guides. Captains of large fishing vessels
and their crew attempt to snare many fish in big waters. They cast nets of various
size in order to collect some fish while allowing others to swim free. They study
macro-level variables like weather patterns, ocean currents, depth and temperature
readings, feeding and migration patterns of various species of fish, and what kind of
36
environmental variables influence large schools of fish. Their bottom line is impacted
by producing significant results, and they use sophisticated equipment to gain what-
ever advantage is available to bring in a significant haul. They are contracted by
larger agencies interested in the weight and volume of their catch, and the value of
their catch must be greater than the expense of pursuing it lest they lose their jobs.
Their livelihood, the health and well being of their family, and indeed the health
and well being of their community rides on their success. Their behavior is thus
governed largely by these macro-level contingencies. They have acquired specialized
knowledge based on collective data from other large fishing ventures, and they use
this knowledge to secure large contracts and to reduce as much as possible the level
of chance (and accompanying mistakes) in their new venture. They are fishermen.
They know they are fishermen. They are good at what they do, and they have
evidence to support this belief.
Let’s now contrast this with the solitary angler working as a paid guide along
a river or stream in some highly prized piece of trout heaven - for example, Southwest
Montana. Many individuals seek out the guide in hopes of catching their dream fish.
The guide must discern what kind of fish and fishing experience they are looking for,
assess the level of skill they currently posses, and then develop a strategic plan that
will increase the individual’s chance of obtaining her goal based on this information.
The guide is being paid to shoulder the responsibility of matching this particular
angler with her particular skill level and equipment with complimentary zones of the
local river system to obtain the desired outcome of catching the customer’s prized
fish. The guide is paid to be the expert on these waters and these fish. Those
who pay for his service may at times come in with preconceived ideas of where to
go, what equipment to use, and when to use it and the guide must respectfully
and gently guide him or her to redirect that energy in a way that will be most
productive in terms of helping the client achieve their stated goal. The angler’s
livelihood, the health and well being of his family and indeed the health and well
being for a larger community may depend on his success as a guide. His behavior
is governed largely by more micro-level contingencies. He has acquired specialized
knowledge based on the detailed study of variation among individual people, rivers,
time of day, and idiosyncratic ecological events such as insect hatchings in order to
best take advantage of inherent chance and mistakes. He is a fisherman. He knows
he is a fisherman. He is good at what he does, and he has evidence to support his
37
belief.
Strange . . . we never lament the gap between commercial fishing fleets
and individual fishing guides. Perhaps it is because they intuitively recognize that
although they both ”fish” their behavior is different. Though the verbiage used
to describe their actions is the same, they pursue different means to different ends.
Vikki Lee (1988) argues convincingly that the proper focus of psychology is not only
on action but also on the fruit of that action. Behavior is defined not by actions
(or means) alone but also by the results (the ends) of those actions. Thus the
proper subject matter of psychology as a scientific discipline is, according to Lee,
the analysis of means-ends units. Behavior is defined not by action alone but by its
means-and-ends.
Researchers are interested in ”fishing” for the variable or variables that pro-
duce the biggest effects in isolation from other variables. Like the fishboat captains
they select their target population and nets of various size to screen out all other
variables in an effort to increase the odds of catching what they seek. These se-
lection measures are a means by which researchers weed out variability in an effort
to discover which variable or predefined combination of variables - occurring in iso-
lation of the random noise generated by life - produces the largest reliable effect
(end). Researchers are interested in minimizing life’s influence to find ”real” (i.e.,
reliable, replicable) effects. Though these ”real” effects may be small for any one
individual entity within a population, there is great value in the cumulative effect
across individuals and across time for the entire population.
Therapist are interested in ”fishing” for the variable or variables that produce
a socially defined important effect for the client sitting across from them at this
moment. Like the fishing guide, they clarify the client’s interest, appraise the skill
level and motivation of the client, take stock of larger ecological variables, and
actively select in all pertinent variables in an effort to increase the odds of the
client catching what they seek. This data gathering is the means by which therapist
increase variability in an effort to discover a combination of variables supported by
the environment and accepted by the client that will produce a previously negotiated
important change (end). Therapist are interested in maximizing life’s influence now
to find ”workable” (i.e., effective, efficient) solutions to address the current crisis.
Striking a healthy balance between processes that minimize variation in some
domains and maximize variation in others is a defining feature of complex adaptive
38
systems (Axelrod & Cohen, 2000). As a complex adaptive system, psychotherapy
continues to change and move forward not merely in spite of but rather as a result
of the creative tension that exists between researchers and therapists. The dynamic
interplay between variation minimizing means-end units (traditionally the domain of
psychological researchers) and variation maximizing means-end units (traditionally
the domain of therapists) functioning within the same system allows the system
to evolve over time. Variation maximizing units prevent stagnation and premature
convergence, while variation minimizing units prevent chaos and eternal boiling.
The field of psychotherapy often errs in believing that services provided by
psychotherapy research and those provided by psychotherapist naturally compete
against one another, and indeed must compete against one another, when in reality
this is not so. As discussed earlier, psychotherapy researchers and psychotherapists
differ in their means-end units. Said differently, they have different customer and
client relationships, and provide distinctly different services. Clients are those that
pay for services. Customers are those who receive services. Like charter fishing boat
captains, psychotherapy researchers are paid by larger entities for accumulated re-
sults. Their customers are often also larger entities interested in accumulated results
(e.g., universities, HMOs, pharmaceutical companies, large groups of psychothera-
pists). Individuals are secondary or tertiary customers. Like private fishing guides,
psychotherapists are paid by individuals for individual results. Their customers are
primarily those individuals who come into their office. Larger entities are secondary
or tertiary customers. The work of one enterprise can and should influence the
other, the output of each provides grist for the other’s mill. These service lines can
be complimentary and work together synergistically, but they likely will not do so
naturally. They will need liaisons who recognize the nature and value of the different
subsystems and can accept the influence of both. Scientist-practitioners are ideally
suited to be these liaisons, and by their training have the opportunity to add unique
value to the system as a whole.
39
Chapter 6
Conclusion
In 1986, Morrow-Bradley and Elliott (1986) conducted a survey of clinical practi-
tioners who were members of the American Psychological Association concerning
the use of psychotherapy research by practicing psychologists. They found that
most practicing psychologists felt: questions addressed in research often are not
clinically relevant, variables studied are often not representative of typical clinical
practice, and forms in which the results are reported (e.g., mean differences, and
F ratios) do not represent clinically important changes or differences. In addition,
the researchers noted that single case research methodology was used infrequently
and practical or relevant measures of psychological change were seldom used. This
study attempted to address some of these concerns.
Reviewing contemporary psychotherapy research in the 1999 Annual Review
of Psychology Kopta, Lueger, Saunders, and Howard (1999) suggested that much
of the clinical research and clinical practice gap might be due to the field’s pref-
erence for randomized clinical trials (RCTs). They proposed that ”this approach
should be replaced by naturalistic designs, which can provide results more appli-
cable to real clinical practice, therefore strengthening external validity” (p. 449).
Essentially, the authors endorsed effectiveness studies as the best means of under-
standing psychotherapy’s impact in real-world clinical settings. However, arguing
that naturalistic designs should replace RCTs misses the critical issue. There is
understanding psychotherapy’s impact in real-world clinical settings, and there is
understanding psychotherapy’s impact on the real world. From a public health
stand point, small, cumulative effects over time are extremely valuable. From an
40
individual’s perspective, relief now (or at least as soon as possible) is highly prized.
The field of psychotherapy is comprised of two different enterprises providing differ-
ent services to different clientele. As such, one does not need to dominate or replace
the other. In fact, the greater system would suffer if one enterprise were to ever
dominate the other. Outcomes of one enterprise can, and often do, serve as fuel - as
generating factors - for the other. This dynamic process of creative tension between
psychotherapy research and psychotherapy practice drives the forward progress of
both enterprises.
Effective psychotherapy is a complex adaptive system. Efficacy studies can
serve to provide guidance on what variables may be important for the therapist to
consider, where the therapist should begin, and how the therapist should proceed in
order to increase the likelihood that a successful outcome will be obtained. Without
this guidance, too much variability would exist within the therapy system making a
positive outcome unlikely (a condition referred to by some theorist as eternal boil-
ing). However, use of empirically supported and empirically validated treatments
are not the same as providing effective treatment. Use of empirically supported
treatment without an appreciation of the unique nuances inherit within each ther-
apeutic relationships, and without tailoring the intervention to fit the context of
the individual client would result in too little variability in the therapy system (a
condition often referred to as premature convergence). The intervention would feel
canned, and a positive outcome would be unlikely.
What is efficacious is not always effective. That is, what is statistically sig-
nificant on a large N basis is not always clinically significant on an individual basis.
What is effective is not always efficacious. A novel, paradoxical intervention may
yield clinically significant change for one client, but may not yield statistically sig-
nificant change for a population of subjects. Efficacy and effectiveness research are
necessary but not sufficient components of evidence based psychotherapy (EBT).
Ideally, the practice of evidenced based psychotherapy results in a whole that is
greater than the some of its parts. Scientist-practitioners have the training and ex-
pertise to distinguish between the two service enterprises within the broader domain
of psychotherapy, and are thus in the unique position to provide valuable feedback
to both. We should focus not on debating which approach is better, but on com-
municating the advantages and necessity of each enterprise to those that reside on
the other side of the gap. We should focus not on closing the gap, but on being the
41
vehicles of trade that transport valuable discoveries from one side to the other. It is
the author’s hope that this journey has provided something of value to both sides
of the divide.
42
References
Abidin, R. (1995). Parenting stress index. Lutz, FL.
Axelrod, R., & Cohen, M. (2000). Harnessing complexity: Organization implications
of a scientific frontier. The Free Press.
Baker, B., & McCurry, M. (1984). School-based parent training: An alternative
for parents predicted to demonstrate low-teaching proficiency following group
training. Education and Training of the Mentally Retarded, 19, 261-267.
Barlow, D. (1981). On the relation of clinical research to clinical practice: Current
issues. Journal of Consulting and Clinical Psychology, 49, 147-155.
Barlow, D. (1996). Health care policy, psychotherapy research, and the future of
psychotherapy. American Psychologist, 51 (1050-1058).
Bauman, K., Reiss, M., Rogers, R., & Bailey, J. (1983). Dining out with children:
Effectiveness of a parent advice package on pre-meal inappropriate behavior.
Journal of Applied Behavior Analysis, 16, 55-68.
Bigelow, K., & Lutzker, J. (2000). Training parents reported for or at risk for child
abuse and neglect to identify and treat their children’s illnesses. Journal of
Developmental and Behavioral Pediatrics, 15, 356-370.
Bor, W., Sanders, M., & Markie-Dadds, C. (2002). The effects of the triple p -
positive parenting program on preschool children with co-occurring disruptive
behavior and attentional/hyperactive difficulties. Journal of Abnormal Child
Psychology.
43
Brightman, R., Baker, B., Clark, D., & Ambrose, S. (1982). Effectiveness of alterna-
tive parent training formats. Journal of Behavior Therapy and Experimental
Psychiatry, 13, 113-117.
Chamberlain, P., & Reid, J. (1987). Parent observation and report of child symp-
toms. Behavioral Assessment, 9, 97-109.
Christophersen, E. (1982). Little people: Guidelines for common sense child rearing
(2nd ed.). Austin, TX: Pro-Ed.
Christophersen, E., & Mortweet, S. (2003). Parenting that works: Building skills
that last a lifetime. Washington, DC: American Psychological Association.
Clark, L. (1991). The video sos: Help for parents. Video recording, Parents Press.
(Bowling Green, KY)
Davis, J., & Daly, D. (2003). Girls and boys town long-term residential program
training manual. Boys Town, NE: Boys Town Press.
Day, D., & Roberts, M. (1983). An analysis of the physical punishment component
of a parent training program. Journal of Abnormal Child Psychology, 11,
141-152.
Engelmann, S., & Carnine, D. (1991). Theory of instruction: Principles and appli-
cations. Eugene, OR: ADI Press.
Eyberg, S. (1988). Parent-child interaction therapy: integration of traditional and
behavioral concerns. Child and Family Behavior Therapy, 10 (1), 33-46.
Eyberg, S., & Ross, A. (1978). Assessment of child behavior problems: The valida-
tion of a new inventory. Journal of Clinical Psychology, 16, 113-116.
Forehand, R. (1986). Parental positive reinforcement with defiant children: Does it
make a difference? Child and Family Behavior Therapy, 8, 19-25.
Forehand, R., & King, H. (1977). Noncompliant children: Effects of parent training
on behavior and attitude change. Behavior Modification, 1 (1), 93-108.
44
Forehand, R., & Long, N. (1996). Parenting the strong willed child: The clinically
proven five-week program for parents of 2-6 year olds. Chicago, IL: Contem-
porary Books.
Forgatch, M., & Toobert, D. (1979). A cost effective parent training program for use
with normal preschool children. Journal of Psychological Practice, 4, 129-145.
Friman, P. (2000). Behavioral, family-style residential care for troubled out-of-
home adolescents: Recent findings. In J. Austin & J. Carr (Eds.), Handbook
of applied behavior analysis (p. 187-210). Reno, NV: Context Press.
Giebenhain, J., & O’Dell, S. (1984). Evaluation of a parent-training manual for
reducing children’s fears of the dark. Journal of Applied Behavior Analysis,
17, 121-125.
Gottman, J. M., & DeClaire, J. (1998). Raising an emotionally intelligent child.
New York, NY: Simon & Schuster.
Graziano, A., & Diament, D. (1992). Parent behavioral training: An examination
of the paradigm. Behavior Modification, 16 (1), 3-38.
Green, R., Hardison, W., & Greene, B. (1984). Turning the table on advice programs
for parents: Using placemats to enhance family interaction at restaurants.
Journal of Applied Behavior Analysis, 17, 497-508.
Hamilton, S., & MacQuiddy, S. (1984). Self-administered behavioral parent training:
Enhancement of treatment efficacy using a time-out signal seat. Journal of
Consulting and Clinical Psychology, 13, 61-69.
Handwerk, M., Field, C., & Friman, P. (2000). The iatrogenic effects of group inter-
vention for antisocial youth: Premature extrapolations. Journal of Behavioral
Education, 10 (4), 223-238.
Hobbs, S., Walle, D., & Caldwell, S. (1984). Maternal evaluation of social reinforce-
ment and time-out: Effects of brief parent training. Journal of Consulting and
Clinical Psychology, 52, 135-136.
Hudson, A. (1982). Training parents of developmentally handicapped children: A
component analysis. Behavior Therapy, 13, 325-333.
45
Jacobson, N., & Truax, P. (1991). Clinical significance: A statistical approach to
defining meaningful change in psychotherapy research. Journal of Consulting
and Clinical Psychology, 59, 12-19.
Kazdin, A. (1987). Treatment of antisocial behavior in children: Current status and
future directions. Psychological Bulletin, 102, 187-203.
Kazdin, A. (1997). Practitioner review: Psychosocial treatments for conduct disor-
der in children. Journal of Child Psychology and Psychiatry, 38, 161-178.
Keating, J., Butz, R., Burke, E., & Heimberg, R. (1983). Dry bed training without a
urine alarm: Lack of effect of setting and therapist contact with child. Journal
of Behavior Therapy and Experimental Psychiatry, 14, 109-115.
Kendall, P., & Grove, W. (1988). Normative comparisons in therapy outcome.
Behavioral Assessment, 10, 147-158.
Kirigin, K., & Wolf, M. (1998). Application of the teaching-family model to children
and adolescents with conduct disorder. In V. VanHasselt & M. Hersen (Eds.),
Handbook of psychological treatment protocols for children and adolescents (p.
359-380). Lawrence Erlbaum Associates.
Kopta, S., Lueger, R., Saunders, S., & Howard, K. (1999). Individual psychotherapy
outcome and process research: Challenges leading to greater turmoil or a
positive transition? In J. Spence, J. Darley, & D. Foss (Eds.), Annual review
of psychology (Vol. 50, p. 441-470). Palo Alto, CA: Annual Reviews.
Latham, G. (1994). The power of positive parenting: A wonderful way to raise
children (Revised ed.). North Logan, UT: P&T Ink.
Lee, V. (1988). Beyond behaviorism. Hillsdale, NJ: Lawrence Erlbaum Associates.
McIntire, R. (1996). Enjoy successful parenting: Practical strategies for parents of
children 2-12 (Revised ed.). Columbia, MD: Summit Crossroads Press.
McMahon, R., & Wells, K. (1998). Conduct problems. In E. Mash & R. Barkley
(Eds.), Treatment of childhood disorders (2nd ed., p. 111-207). New York:
Guilford.
46
Miller, G., & Prinz, R. (2003). Engagement of families in treatment for childhood
conduct problems. Behavior Therapy, 34, 517-534.
Morrow-Bradley, C., & Elliott, R. (1986). Utilization of psychotherapy outcome
research by practicing psychotherapists. American Psychologist, 41, 188-197.
Nathan, P., & Gorman, J. (2002). Efficacy, effectiveness, and the clinical utility of
psychotherapy research. In P. Nathan & J. Gorman (Eds.), Treatments that
work (2nd ed., p. 643-654). New York: Oxford Press.
O’Dell, S. (1974). Training parents in behavior modification: A review. Psychological
Bulletin, 81 (7), 418-433.
O’Dell, S. (1985). Progress in parent training. Progress in Behavior Modification,
19, 57-108.
O’Dell, S., O’Quin, J., Alford, B., O’Briant, A., Bradlyn, A., & Giebenhain, J.
(1982). Predicting the acquisition of parenting skills via four training methods.
Behavior Therapy, 13, 194-208.
Ogles, B., Lambert, M., & Masters, K. (1996). Assessing outcome in clinical
practice. Boston: Allyn and Bacon.
Patterson, G. (1971). Families: Applications of social learning to family life. Cham-
paign, IL: Research Press.
Patterson, G., & Gullion, M. (1968). Living with children: New methods for parents
and teachers. Champaign, IL: Research Press.
Patterson, G., Littman, R., & Hinsey, W. (1964). Parental effectiveness as rein-
forcers in the laboratory and its relation to child rearing practices and adjust-
ment in the classroom. Journal of Personality, 32, 180-199.
Pevsner, R. (1982). Group parent training versus individual family therapy: An
outcome study. Journal of Behavior Therapy and Experimental Psychiatry,
13, 119-122.
Phelan, T. (2001). 1-2-3 magic: Managing children’s difficult behavior. Video
recording, Child Management Inc. (Glen Ellyn, IL)
47
Prieto-Bayard, M., & Baker, B. (1986). Parent training for spanish speaking families
with a retarded child. Journal of Community Psychology, 14, 134-143.
Reid, J. (1993). Prevention of conduct disorder before and after school entry: Relat-
ing interventions to developmental findings. Development and Psychopathol-
ogy, 5, 243-262.
Rickert, V., Sottolano, D., Parrish, J., Riley, A., Hunt, F., & Pelco, L. (1988). Train-
ing parents to become better behavior managers: The need for a competency-
based approach. Behavior Modification, 12 (4), 475-496.
Roberts, M. (1985). Praising child compliance: Reinforcement or ritual? Journal
of Abnormal Child Psychology, 13, 611-629.
Roberts, M. (1988). Enforcing chair timeouts with room timeouts. Behavior Modi-
fication, 12, 353-370.
Robinson, E., Eyberg, S., & Ross, A. (1980). The standardization of an inven-
tory of child conduct problem behaviors. Journal of Consulting and Clinical
Psychology, 9, 22-28.
Ruma, P., Burke, R., & Thompson, R. (1996). Group parent training: Is it effective
for children of all ages? Behavior Therapy, 27, 159-169.
Sanders, M. (1996). Every parent’s survival guide. Video recording, Families Inter-
national Pty. (Milton, Qld, Australia)
Walle, D., Hobbs, S., & Caldwell, H. (1984). Sequencing of parent training proce-
dures: Effects on child noncompliance and treatment acceptability. Behavior
Modification, 8, 540-552.
Webster-Stratton. (1992). The incredible years: A trouble shooting guide for parents
of children age 3-8. Video series, Umbrella Press. (Toronto)
Webster-Stratton, C. (1981). Modification of mothers’ behaviors and attitudes
through a videotape modeling group discussion program. Behavior Therapy,
12, 634-642.
48
Webster-Stratton, C. (1984). Randomized trial of two parent training programs for
families with conduct disordered children. Journal of Consulting and Clinical
Psychology, 52, 666-678.
Webster-Stratton, C. (1996). Early intervention with videotape modeling: Pro-
grams for families of children with oppositional defiant disorder or conduct
disorder. In E. HIbbs & P. Jensen (Eds.), Psychosocial treatments for child
and adolescent disorders: Empirically based strategies for clinical practice (p.
435-474). Washington, D.C.: American Psychological Association.
Webster-Stratton, C., Hollinsworth, T., & Kolpacoff, M. (1989). The long term
effectiveness and clinical significance of three cost effective training programs
for families with conduct problem children. Journal of Consulting and Clinical
Psychology, 57 (4), 550-553.
Webster-Stratton, C., Kolpacoff, M., & Hollinsworth, T. (1988). Self-administered
videotape therapy for families with conduct problem children: Comparison
with two cost effective treatments and a control group. Journal of Consulting
and Clinical Psychology, 56 (4), 558-566.
49
Appendix A
Figures
50
03-06 03-11 03-16 03-21 03-26 03-31 04-05 04-10 04-15 04-200
5
10
15
20
25
30
Tot. Beh.Tar. Beh.
PDR Scores - Cohort 1
Subjects 03, 05, and 02
03-06 03-11 03-16 03-21 03-26 03-31 04-05 04-10 04-15 04-200
5
10
15
20
25
30
03-06 03-11 03-16 03-21 03-26 03-31 04-05 04-10 04-15 04-200
5
10
15
20
25
30
Subject 03
Subject 05
Subject 02
Figure A.1: PDR Data Cohort 1
51
Baseline1 Baseline2 Post-Tx05
10152025303540455055606570
T-S
core
Intensity
Problem
ECBI T-Scores - Cohort 1Subjects 03, 05, and 02
Baseline1 Baseline2 Post-Tx05
10152025303540455055606570
T-S
core
Baseline1 Baseline2 Post-Tx05
10152025303540455055606570
T-S
core
Subject 003
Subject 005
Subject 002
Figure A.2: ECBI Data Cohort 1
52
Pre-Tx Post-Tx3035404550556065707580859095
100
Perc
entil
e
LSTSCDPD
PSI Percentile Scores - Cohort 1Subjects 03, 05, and 02
Pre-Tx Post-Tx3035404550556065707580859095
100
Perc
entil
e
Pre-Tx Post-Tx3035404550556065707580859095
100
Perc
entil
e
Subject 03
Subject 05
Subject 02
Figure A.3: PSI Data Cohort 1
53
03-06 03-11 03-16 03-21 03-26 03-31 04-05 04-10 04-15 04-20 04-250
5
10
15
20
25
30#
Beh
avio
rs E
ndor
sed
Tot. Beh.Tar. Beh.
PDR Data - Cohort 2Subjects 01 and 06
03-06 03-11 03-16 03-21 03-26 03-31 04-05 04-10 04-15 04-20 04-250
5
10
15
20
25
30
# B
ehav
iors
End
orse
d
Subject 01
Subject 06
Figure A.4: PDR Data Cohort 2
54
Baseline1 Baseline2 Post-Tx05
101520253035404550556065707580
T-S
core
IntensityProblem
ECBI T-Scores - Cohort 2Subjects 01 and 06
Baseline1 Baseline2 Post-Tx05
101520253035404550556065707580
T-S
core
Subject 01
Subject 06
Figure A.5: ECBI Data Cohort 2
55
Pre-Tx Post-Tx20253035404550556065707580859095
100
Perc
entil
e
LSTSCDPD
PSI Percentile Scores - Cohort 2
Subjects 01 and 06
Pre-Tx Post-Tx20253035404550556065707580859095
100
Perc
entil
e
Figure A.6: PSI Data Cohort 2
56
04-13 04-20 04-27 05-04 05-11 05-18 05-25 06-01 06-080
5
10
15
20
25
30#
Beh
avio
rs E
ndor
sed
Tot. Beh.Tar. Beh.
PDR Data - Cohort 3Subjects 08, 11, and 10
04-13 04-20 04-27 05-04 05-11 05-18 05-25 06-01 06-080
5
10
15
20
25
30
# B
ehav
iors
End
orse
d
04-13 04-20 04-27 05-04 05-11 05-18 05-25 06-01 06-080
5
10
15
20
25
30
# B
ehav
iors
End
orse
d
Subject 010
Subject 011
Subject 008
Figure A.7: PDR Data Cohort 3
57
T1 T2 T305
1015202530354045505560657075
T-S
core
IntensityProblem
ECBI Data - Cohort 3Subjects 08, 11, and 10
T1 T2 T305
1015202530354045505560657075
T-S
core
T1 T2 T305
1015202530354045505560657075
T-S
core
Subject 08
Subject 11
Subject 10
Figure A.8: ECBI Data Cohort 3
58
Pre Post20253035404550556065707580859095
100
Perc
entil
e
LSTSCDPD
PSI Data - Cohort 3Subject 08, 11, and 10
Pre Post20253035404550556065707580859095
100
Perc
entil
e
Pre Post20253035404550556065707580859095
100
Perc
entil
e
Subject 08
Subject 11
Subject 10
Figure A.9: PSI Data Cohort 3
59
0 5 10 15 20Unadjusted Baseline Mean
0
5
10
15
20T
reat
men
t Mea
n
PDR Total - Reliable Change Index (RCI)
With and Without Baseline Mean Adjustment
0 5 10 15 20Adjusted Baseline Mean
0
5
10
15
20
Tre
atm
ent M
ean
Figure A.10: PDR Total Behavior Score RCI
60
100 110 120 130 140 150 160 170 180 190 200
Intensity score at T1
100
110
120
130
140
150
160
170
180
190
200
Inte
nsity
sco
re a
t T2
ECBI - Reliable Change Index
Intensity and Problem Scales T1 to T2
0 5 10 15 20 25 30
Problem score at T1
0
5
10
15
20
25
30
Prob
lem
sco
re a
t T2
Figure A.11: ECBI: RCI T1 to T2
61
100 110 120 130 140 150 160 170 180 190 200
Intensity score at T2
100
110
120
130
140
150
160
170
180
190
200
Inte
nsity
sco
re a
t T3
ECBI - Reliable Change Index (RCI)
Intensity and Problem Scales T2 to T3
0 5 10 15 20 25 30
Problem score at T2
0
5
10
15
20
25
30
Prob
lem
sco
re a
t T3
Figure A.12: ECBI: RCI T2 to T3
62
100 110 120 130 140 150 160 170 180 190 200
Intensity score at T1
100
110
120
130
140
150
160
170
180
190
200
Inte
nsity
sco
re a
t T3
ECBI - Reliable Change Index (RCI)
Intensity and Problem Scales T1 to T3
0 5 10 15 20 25 30
Problem score at T1
0
5
10
15
20
25
30
Prob
lem
sco
re a
t T3
Figure A.13: ECBI: RCI T1 to T3
63
175 200 225 250 275 300 325 350 375 400Total Stress score: Pre-treatment
175
200
225
250
275
300
325
350
375
400
Tot
al S
tres
s sc
ore:
Pos
t-tr
eatm
ent
PSI Reliable Change Index (RCI)
Total Stress and Parent Domain Scores
80 90 100 110 120 130 140 150 160 170 180 190 200Parent Domain score: Pre-treatment
80
90
100
110
120
130
140
150
160
170
180
190
200
Pare
nt D
omai
n Sc
ore:
Pos
t-tr
eatm
ent
Figure A.14: PSI: RCI Total Stress and Parent Domain
64
Appendix B
Tables
1Webster-Stratton (1982), GDVM = Group Discussion plus Video Modeling2Webster-Stratton (1988), IVM = Individually Administered Video Modeling3Webster-Stratton (1988), Cont= Control group4Data for Subjects 8-11 are presented in Table 55Webster-Stratton (1988), IVM = Individually Administered Video Modeling6Webster-Stratton (1988), Cont= Control group7Based on Adjusted Baseline Mean
65
Table B.1: CSP Learn At Home Video SessionsVideotape Lesson Content
Video 1 1 Parents Are TeachersDisciplineClear communicationUsing consequences
2 Encouraging Positive BehaviorGeneral praise and affectionGiving specific praiseUsing charts and contracts
3 Preventing MisbehaviorUsing preventive teachingTeaching social skills
Video 2 4 Correcting Problem BehaviorsUsing corrective teachingTeaching positive alternative skills
5 Handling Emotionally Intense InteractionsStaying calm when children push buttonsTeaching emotional expressionTeaching self-control
6 Helping Children Succeed in SchoolEstablishing homework timeTeaching social skills for schoolGetting involved with schoolCommunicating with school staff
Table B.2: PDR Means by Subject for Baseline and Treatment ConditionsMean Behaviors Endorsed
x̄ Total x̄ TargetSubject Baseline Treatment Change Baseline Treatment Change
S001 11.67 10.91 -0.76 3.00 2.13 -0.87S002 16.88 15.85 -1.03 3.00 2.85 -0.15S003 11.75 7.24 -4.51 3.00 2.29 -0.71S005 7.00 4.78 -2.22 2.17 0.74 -1.43S006 12.13 9.75 -2.38 4.25 2.50 -1.75S008 8.43 9.08 +0.65 2.33 2.08 -0.25S010 16.00 11.32 -4.68 4.00 3.23 -0.77S011 13.83 12.47 -1.36 3.83 2.76 -1.07
66
Table B.3: ECBI Raw Scores Pre- and Post-TreatmentSubject Scale Time 1 Time 2 Time 3 T1-T2 T2-T3
S001 Intensity 172 155 169 -17 +14Problem 23 27 26 +4 -1
S002 Intensity 140 138 132 -2 -6Problem 21 21 16 0 -5
S003 Intensity 121 114 122 -7 +8Problem 14 10 8 -4 -2
S005 Intensity 111 116 112 +5 -4Problem 14 16 13 +2 -3
S006 Intensity 128 135 116 +7 -19Problem 15 11 6 -4 -5
S008 Intensity 121 126 120 +5 -6Problem 5 5 5 0 0
S010 Intensity 178 160 143 -18 -17Problem 26 24 24 2 0
S011 Intensity 143 178 126 +35 -52Problem 12 18 13 +6 -5
WS1982 Intensity 118.5 106.68 -11.82(GDVM)1 Problem 7.9 4.2 -3.70
WS1988 Intensity 156.07 126.15 -29.92(IVM)2 Problem 20.11 11.70 -8.41
WS1988 Intensity 157.25 147.59 -9.66(Cont)3 Problem 22.55 19.14 -3.41
67
Table B.4: PSI Raw Domain Scores Pre- and Post-Treatment (Subjects 1-6)
Subjects 01 - 06 4
Subject Domain Pre Post Change
S001 Life Stress 16 21 +5Total Stress 279 257 -22
Child Domain 130 128 -2Parent Domain 149 129 -20
S002 Life Stress 53 42 -11Total Stress 295 338 +43
Child Domain 131 159 +28Parent Domain 164 179 +15
S003 Life Stress 9 9 0Total Stress 234 252 +18
Child Domain 115 114 -1Parent Domain 119 138 +19
S005 Life Stress 13 11 -2Total Stress 262 246 -16
Child Domain 115 103 -12Parent Domain 147 143 -4
S006 Life Stress 6 6 0Total Stress 249 237 -12
Child Domain 108 106 -2Parent Domain 141 131 -10
68
Table B.5: PSI Raw Domain Scores Pre- and Post-Treatment (Subjects 8-11)Subjects 08 - 11
Subject Domain Pre Post Change
S008 Life Stress 7 7 0Total Stress 205 208 +3
Child Domain 93 93 0Parent Domain 112 115 +3
S010 Life Stress 23 25 +2Total Stress 335 315 -20
Child Domain 156 145 -11Parent Domain 179 170 -9
S011 Life Stress 6 6 0Total Stress 213 238 +25
Child Domain 107 103 -4Parent Domain 106 135 +29
IVM5 Parent Domain 151.44 139.48 -11.96Cont6 Parent Domain 141.66 138.03 -3.63
Table B.6: PDR Reliable Change Index (RCI)
Subject RCI RCIa7
S001 -0.57 -0.06S002 -0.78 -1.09S003 -3.43 -2.34S005 -1.63 -0.87S006 -1.81 -1.49S008 1.29 2.11S010 -3.56 -3.42S011 -1.03 -1.31
69
Table B.7: ECBI Reliable Change Index (RCI)
RCI scores from:Subject Scale T2 to T1 T3 to T2 T3 to T1
S001 Int -1.29 1.06 -0.23Prob 1.50 -0.37 1.12
S002 Int -0.15 -0.46 -0.61Prob 0 -1.87 -1.87
S003 Int -0.53 0.61 0.08Prob -1.50 -0.75 -2.25
S005 Int 0.38 -0.30 0.08Prob 0.75 -1.12 -0.37
S006 Int 0.53 -1.44 -0.91Prob -1.50 -1.87 -3.37
S008 Int 0.38 -0.46 -0.08Prob 0 0 0
S010 Int -1.37 -1.29 -2.66Prob -0.75 0 -0.75
S011 Int 2.66 -3.95 -1.29Prob 2.247 -1.87 0.38
Table B.8: PSI Reliable Change Index (RCI)Subject Scale RCI
S001 Total Stress -2.69Parent Domain -3.10
S002 Total Stress 5.25Parent Domain 2.32
S003 Total Stress 2.20Parent Domain 2.94
S005 Total Stress -1.96Parent Domain -0.62
S006 Total Stress -1.47Parent Domain -1.55
S008 Total Stress 0.37Parent Domain 0.46
S010 Total Stress -2.44Parent Domain -1.39
S011 Total Stress 3.05Parent Domain 4.49
70
Appendix C
Consumer Satisfaction Survey -
Comments
C.1 What tips and suggestions from the videotapes did
you attempt to use in your home?
• Tell how I want the child to act instead of yelling at him
• Get a chart and mark when child has done what he was supposed to do, after
a week if child has one or two or whatever then he can pick a treat accordingly
• Preventive teaching, proactive teaching along with consequences. Getting
more involved with his teachers and his learning.
• Using more positive praise, using specifics when telling children to stop un-
wanted behaviors, using the 4-step process in highly emotional situations
• Staying calm, consequences, siblings taking breaks from one another when
fighting with each other
• Identify what my child was doing wrong. Consequences - positive and negative.
• Preventive teaching
• Correcting problem behaviors using time-out and then tell him what is wrong
and what is right thing to do and practice.
71
• Separate siblings when they are arguing.
• Talking positively
• Describe behavior, use age appropriate reason, have child practice the right
thing
• Have the child say ”OK” and reward good behavior
C.2 What tips and suggestions did you find most help-
ful?
• Tell child how I want him to act
• Use charts to show what needs to be done
• Have a chore jar and joy jar
• Give praise for good behavior
• Staying calm and identifying what he is doing wrong, rather than just yelling.
Preventive teaching. Using a school note with his teachers.
• Staying calm, being specific about unwanted behaviors, giving clear instruc-
tions
• Giving consequences when you and your child are calm, otherwise nothing gets
solved
• Following through with consequences when able
• Corrective teaching, joy jar/job jar, positive and negative consequences, time-
outs
• Calming the child down and then talking with him about the problem behavior
• All tips are helpful, but some are used more than others.
• Provided more options/choices for what to do to change behavior other than
time-out, spanking, or slapping.
72
C.3 What tips and suggestions did you find least help-
ful?
• Suggestions dealing with teens. My children are still young.
• Job and joy jars because these are for older children.
• The school section, only because it hasn’t been an issue or wasn’t new material
• Trying to teach to children before a situation occurs
• Parts of segment 5
• All are helpful
• None
• None
C.4 What suggestions would you have to improve the
videotape training system?
• Not sure, I thought this was very good. I liked the book and reading, (better)
than watching video.
• Using a more realistic approach to handling out of control children.
• None
• I think the video segment about staying calm should be 1st or 2nd not 5th.
I think that is one of the hardest things to do as a parent when dealing with
misbehaviors in children. After that, you can teach.
• None
• More situations need to be considered, such as dealing with conflict between
friends or classmates, setting up a sleeping and eating schedule, etc.
• None.
• More suggestions on teaching social skills.
73
C.5 Would you recommend the ”Common Sense Par-
enting Learn-at-Home Kit” to a friend or family
member who was having difficulty managing their
child’s behavior? Why or why not?
• Yes, because some of the things you don’t normally think of unless you’ve had
a dozen kids. First time parents need all the help they can get. And this is a
good, helpful way of learning.
• Yes, because everyone can learn something from these tapes on how to handle
your child’s behaviors and use them in their own way to create their own style.
• Yes, it is easy to watch in the short segments and the examples are helpful
• Yes, it had a lot of helpful items in it. It at least gives a parent something to
trying in dealing with problems.
• Yes, it’s easy to follow. You are able to do it at your own pace. It is a great
reference to have on hand.
• Yes, I would (definitely) because there are so many tips that we can learn
from there. Those suggestions make our life a lot easier and give us a new
view about children’s behavior problems. That is, they need to learn and we
need to teach.
• Sure, everyone needs a little help perfecting their parenting skills.
• Yes, it gives you a starting point, and some tools to use.
C.6 On a scale of 1 (very unsatisfied) to 10 (very sat-
isfied) how would you rate your experience as a
participant in this study?
• 10
• 6, because I think my child has deeper issues than these steps can help with.
74
• 8
• 10
• 10
• 1 (Author’s note: Due to extremely positive comments about the CSP Learn-
at-Home Kit on the other items, I suspect this subject meant to rate their
experience as a ”10” not a ”1”.)
• 8
• 8
75
Appendix D
Parent Daily Report (PDR)
Items
In the last 24-hours has (child’s name) . . .
1. been aggressive?
2. argued?
3. wet the bed?
4. been overly competitive?
5. complained?
6. cried?
7. been defiant?
8. been destructive?
9. been overly fearful?
10. fought with siblings?
11. set things on fire?
12. hit others?
76
13. been hyperactive?
14. been irritable?
15. told a fib or lied?
16. been negative?
17. been noisy?
18. been non-compliant?
19. refused to eat meals?
20. wet his/her pants?
21. pouted?
22. run around like a wild child?
23. run away from you?
24. been overly sad?
25. soiled his/her pants?
26. stolen something?
27. talked back to an adult?
28. teased someone?
29. thrown a temper tantrum?
30. whined?
31. yelled?
32. had contact with the police?
33. had behavior problems at school or daycare?
77
Vita
Sean Smitham was born in San Antonio, TX in 1973 and he spent most of his
early life in Western Montana. Always interested in positively impacting the lives of
young people, Sean served as a mentor and peer counselor in high school graduating
from Butte High School in 1992. He further indulged his passion for helping others
by working as a Resident Advisor (RA) and Resident Director (RD) at Montana
State University — Bozeman where he graduated with honors in 1996 earning his
Bachelor of Science degree in Psychology.
Sean was accepted into the Clinical Psychology program at Western Michigan
University (WMU) in the fall of 1998 and earned his Master’s degree in 2002. His
practicum experiences at WMU included time spent providing individual and family
therapy services to youth through Borgess Hospital’s Adolescent Inpatient and Par-
tial Hospitalization programs. In addition, Sean served as the Student Coordinator
of WMU’s Clinical Psychology Training Clinic from 2001 to 2003. He completed an
American Psychological Association (APA) approved predoctoral internship with
the Nebraska Internship Consortium in Professional Psychology (NICPP) in July
2004. During his internship year, Sean worked under the auspices of the Specialized
Clinical Services and Research department providing psychological services for the
Family Home program at Girls and Boys Town in Boys Town, NE.
Sean is currently receiving advanced training in behavioral pediatrics as a
postdoctoral fellow for the Girls and Boys Town Behavioral Pediatrics and Family
78
Services Clinic where he provides direct clinical care to children and their families
under the tutelage of Pat Friman.
Permanent Address: 13603 Flanagan Blvd., STOP: 18150
Boys Town, NE 68010
USA
This dissertation was typeset with LATEX2ε1 by the author.
1LATEX2ε is an extension of LATEX. LATEX is a collection of macros for TEX. TEX is a trademarkof the American Mathematical Society. The macros used in formatting this dissertation werewritten by Dinesh Das, Department of Computer Sciences, The University of Texas at Austin, andextended by Bert Kay, James A. Bednar, and Ayman El-Khashab. Minor changes to meet WMUrequirements were provided by Sean Smitham.
79
Recommended