2 Summary of Presentation Evaluating Interventions
Recommendations for Parents Fads
Slide 3
What Interventions Should We Use? The Right to Effective
Behavioral Treatment ABA Task Force (1987) Published in JABA (1988)
http://www.abainternational.org/ABA/statements/trea tment.asp
http://www.abainternational.org/ABA/statements/trea tment.asp
Individuals who are recipients of treatment designed to change
their behavior have the right to: A therapeutic environment
Services whose overriding goal is personal welfare Treatment by a
competent behavior analyst Programs that teach functional skills
Behavioral assessment and ongoing evaluation The most effective
treatment procedures available
Slide 4
The Most Effective Treatment Procedures Available An individual
is entitled to effective and scientifically validated treatment; In
turn, the behavior analyst has an obligation to use only those
procedures demonstrated by research to be effective.
Slide 5
Behavior Analyst Certification Board (BACB) Guidelines for
Responsible Conduct Section 2.09 Treatment Efficacy
http://www.bacb.com/consum_frame.html The behavior analyst always
has the responsibility to recommend scientifically supported most
effective treatment procedures. Effective treatment procedures have
been validated as having both long-term and short-term benefits to
clients and society. Clients have a right to effective treatment
(i.e., based on the research literature and adapted to the
individual client). Behavior analysts are responsible for review
and appraisal of likely effects of all alternative treatments,
including those provided by other disciplines and no
intervention.
Slide 6
As Behavior Analysts, we Maintain a healthy skepticism
Differentiate opinions, beliefs, and speculations from facts Dont
make claims without supporting objective data Skepticism cynicism
Being open and being skeptical might seem contradictory, but theyre
not. This is the way to think scientifically. Some phenomena that
seem outlandish are valid Lucid dreaming, extraordinary feats of
human memory and appropriate clinical uses of hypnosis (as opposed
to the scientifically unsupported use of hypnosis for memory
recovery). We must keep our minds open but not so open that our
brains fall out
Slide 7
Ghezzi, Williams, and Carr (1999) Preface Those who fall in
love with practice without science are like a sailor who enters a
ship without a helm or compass, and who never can be certain
whither he is going (Leonardo da Vinci)
Slide 8
Green (1996) Behavior analysts use a number of practices to
increase the objectivity of evidence about treatment effects
Operational definitions (specific and observable descriptions of
behavior) Measurement procedures are clearly specified
Interobserver agreement data are collected Multiple measures of the
effects of tx can be obtained for example, Direct observation
Standardized tests Parent ratings of behavior
Slide 9
How Do Parents Choose a Treatment for Their Child with Autism?
Recommendation by pediatrician or other doctor School Other parent
Internet, book Do you think these sources reliably recommend
interventions based on the objectivity of the evidence?
Slide 10
Fads in Autism Treatment Metz, Mulick, and Butter (2005):
Google search - autism and treatment = 65 distinct interventions
sold as effective for treating autism Telepathy, injection of sheep
stem cells, thyme, swimming with dolphins Fads in the media In
autism treatment, fads tend to be harmful Waste time Waste money
Falsely raise hopes and expectations Distract from effective
treatments In some cases, harm children and families
Slide 11
Why Do Ineffective/Unproven Treatments Become Fads? (Vyse,
2005) Theyre presented as relatively easy and with immediate
effects by people who appear warm, sincere, and attentive Best
treatment is deemed distasteful or is hard to get Alternate
treatments are supported by popular culture, feel right, seem to
make sense Professionals or other people recommend them Most
professionals are not trained how to evaluate treatments Autism
treatment is a commercial enterprise
Slide 12
Why Do Ineffective/Unproven Treatments Become Fads? (Metz et
al., 2005) Parents are in a vulnerable position Doing SOMETHING
feels better than doing nothing Grieving process Avoidance of guilt
Urgency Qualities that make parents dedicated and enthusiastic make
them vulnerable to accept claims without close scrutiny Nature of
Autism Cause is unknown Autism is mysterious myth of the hidden
inner child
Slide 13
Even Well-Meaning Professionals Use Ineffective/Unproven
Interventions (Metz et al., 2005) Example of facilitated
communication Rationale provided for the therapy may be logical or
sound convincing But many are based on flawed theories about the
cause of autism For the overwhelming majority of autism treatments,
anecdotes and testimonials are the only supporting evidence Almost
none stand up to reasonably rigorous scientific evaluation Many
therapies for autism in widespread use today have been shown to be
ineffective in scientific studies Some have been shown to be
harmful Still others have not been subjected to any rigorous
evaluations
Slide 14
Where can we refer parents? Association for Science in Autism
Treatment www.Asatonline.org
Slide 15
Evidence-Based Practice Evidenced-based interventions
Evidence-based practices Empirically supported treatments Best
practices
Slide 16
Evidence-Based Practice (EBP) ODonohue and Ferguson (2006) the
integration of the best available research with clinical expertise
in the context of patient characteristics, culture, and preferences
(APA Presidential Task Force on Evidence-Based Practice, 2006, p.
273) http://www.apa.org/practice/ebp.html Other similar terms
Empirically-validated therapies (EVT) Empirically-supported
therapies (EST)
Slide 17
Development of EBP ODonohue and Ferguson (2006) 20 years ago -
Clinical practice guidelines developed in medicine to Help
standardize decision-making in treatment Encourage use of
empirically sound treatments Improve quality of medical services
and reduce errors Early 90s American Psychiatric Association
developed their own clinical practice guidelines Intended to assist
psychiatrists in decision-making in treatment Tended to recommend
pharmacotherapy over non- pharm treatments Mid 90s American
Psychological Association (APA) Task Force publishes their own
clinical practice guidelines
Slide 18
The Chambless Criteria Division 12 of the American
Psychological Association (APA) Clinical Psychology Established a
Task Force (headed by Diane Chambless) to identify and promote
empirically supported psychological treatments in 1993. Chambless
et al. (1998) available at
http://www.apa.org/divisions/div12/est/97 report.pdf
http://www.apa.org/divisions/div12/est/97 report.pdf
Slide 19
Slide 20
Slide 21
Weaknesses of Chambless Criteria ODonohue and Ferguson (2006)
Do not take clinical significance into account Focus on efficacy,
not effectiveness Will the same results be obtained in natural
settings? Is the treatment practical to implement in terms of
staffing, funds, expertise? Most studies on the EBP list do not
include participants with comorbid disorders Are biased toward
group designs and inferential statistics Only 2 good group design
exps, but at least 9 single-case exps Many single-case designs dont
compare 2 treatments
Slide 22
Individuals with Disabilities Education Improvement Act (IDEA)
& No Child Left Behind (NCLB) NCLB of 2001: interventions
should be based on scientific research IDEA of 2004: scientifically
based institutional practices for those in spec eds Services in a
childs IEP should be based on peer reviewed research the extent
practicable (IDEA, 2004)
Slide 23
Professional Organizations APA Established scientific and
professional knowledge of the discipline National Association of
School Psychology (NASP) Practice should be based on scitific
research
Slide 24
Horner, Carr, Halle, McGee, Odom, and Wolery (2003)
Single-subject research documents a practice as evidence-based when
The practice is operationally defined The outcomes and context in
which the practice is to be used are defined (target behaviors
affected, setting, age, skills, diagnosis, implementer) The
practice is implemented with fidelity (tx integrity data) The
change is the DV is shown to result from the IV experimental
control! Effects of the practice are replicated across a sufficient
number of studies
Slide 25
Make a table: APANY State Dept of Heath (1999) CECNASPCASP
rating system for single case designs Gina Greens Gold standards
National Standards Projects Well-established Tx: Efficacy
demonstrated (> placebo or = to established Tx) in at least 2
good b/w group designs expos OR large series of s-c designs exps
with good design and comparison to another Tx Probably Efficacios
Tx Experimental Tx
Slide 26
Script and Script Fading APANY StateCECNASPNASP rating system
Gina Greens Gold Standard s NSP
Slide 27
Problems with Current Clinical Guidelines Studies examined were
before 1999 Process used by each of the task forces were not made
public Not comprehensive
Slide 28
National Standards Project -examined empirical evidence
supporting interventions that could be used in school setting for
those younger than 22 Exclusion criteria
Medical/complementary/alternative Axis II outside of MR Studies
that could not be easily performed in schools Qualitative analyses
or published in non-peer reviewed journals
Slide 29
Scientific Merit Rating Scale Rated from 0-5 Research design
Measurement of the DV Measurement of the IV Participant selection
Generalization
Slide 30
Coding Beneficial treatment effects reported Single-case
designs: Strong, moderate weak Functional relationship has been
established & is replicated Pont of comparison across
conditions exists Magnitude of change is consistent Percentage eon
non-overlapping points is impressive No treatment effect reported
Adverse treatment effects reported
Slide 31
National Standards Project
http://www.nationalautismcenter.org/a bout/national.php
Slide 32
References APA Presidential Task Force on Evidence-Based
Practice. (2006). Evidence-based practice in psychology. American
Psychologist, 61, 271- 285. Chambless, D.L., Baker, M., Baucom,
D.H., Beutler, L.E., Calhoun, K.S., Crits-Christoph,P., et al.
(1998). Update on empirically validated therapies, II. The Clinical
Psychologist, 51, 3-16. Ghezzi, P.M., Williams, W.L., & Carr
J.E. (1999). Autism: Behavior analytic perspectives. Reno, NV:
Context Press. Green, G. (1996). Evaluating claims about treatments
for autism. In C. Maurice, G. Green, & S.C. Luce (Eds.),
Behavioral intervention for young children with autism. (pp.
15-28). Austin, TX: Pro-ed. Metz, B., Mulick, J.A., & Butter,
E.M. (2005). Autism: A lat-20 th - century fad magnet. In J.W.
Jacobson, R.M. Foxx, & J.A. Mulick (Eds.), Controversial
therapies for developmental disabilities. (pp. 237-263). Mahwah,
NJ: Lawrence Erlbaum. ODonohue, W., & Ferguson, K.E. (2006).
Evidence-based practice in psychology and behavior analysis. The
Behavior Analyst Today, 7, 335- 350. Vyse, S. (2005). Where do fads
come from? In J.W. Jacobson, R.M. Foxx, & J.A. Mulick (Eds.),
Controversial thearpies for developmental disabilities. (pp. 3-17).
Mahwah, NJ: Lawrence Erlbaum.
