Outcomes and methodological considerations across...

Outcomesandmethodologicalconsiderationsacrossthreecommunity-basedeffectivenesstrialsofASD-focused

interventions

CRIEI2018

SarahRieth,KelseyDickson,LaurenBrookman-Frazee,&Aubyn StahmerCRIEI2018

InterventionwithToddlerswithASD?

Whatworks?(circa2006)

Whatfitsfamiliesneeds?

Whatcan/will

communityprovide?

Training $$$$

CommunityProviders

ResearchTeam

FundingAgency

OccupationalTherapy

InfantMentalHealth

Speech&Language

Medical

Families

Education Behavior

Autism

CBPR

Public(PartC)

PrivateInsurance

ASDexperience

Multi-Cultural

Intervention with Toddlers with ASD?

CommunityProviders

ResearchTeam

FundingAgency Families

Bond * Regulate * Interact * Develop * Guide * Engage

InitialInterventionSelectionProcess

EMT P.L.A.Y. ImPACT

• Parent focused, coaching• Blend of developmental and behavioral strategies (NDBI)

• Local training potential• Range of disciplines• Easy to use materials• Flexibility for agencies and families• Evidence-based

Intervention Selection

Responding to Community Needs

+ =

Responding to Community Needs

EnhancementstoProviderTraining:• Earlysocial-emotionaldevelopment(context)

• Opportunitiesforhands-on-practicethroughout

• Parentengagementfocus(supportcoaching)

• Reflectivepractice• MAlevelpractitioners

AdaptationstoIntervention:• Toddler-ize examples• Emphasisondailyactivities• Rethinking“homework”• Remove“autism”• Smootherintegrationbetweendevelopmentalandbehavioral

Two Recent Studies

TrainTheTrainer–

CommunityCapacity

PilotStudy–Child

Outcomes

Trainers:ModelDeveloper+BRIDGEMentors

Trainers:BRIDGEMentors

PriorStudy– 4agencies,8providers

Train the Trainer Design

CurrentStudy– 14agencies,14leaders,43providers

• Multiple baseline across groups– 3 groups, 4-5 agencies per group

• Part 1: Agency Training – by BRIDGE Mentors– 3 months training, 3 months on-going support– Group A: Agencies with prior involvement– Group B: Community agencies – San Diego– Group C: Community agencies – Sacramento (shortest BL)

• Part 2: Provider Training – by Agency Trainers– Materials provided by research team– Training conducted independently– 3 months training, 3 months on-going support … sort of

Train the Trainer Design

• Measures:– Demographics– Intervention fidelity (10 min sample)– Coaching strategies fidelity– Implementation Survey– Organizational Readiness to Change Survey– Self-rated training fidelity (for Agency Trainers)

Train the Trainer Design

9 10

Training - 3 months

Initial ½ Day OverviewEarly Development

(4 hours)

Coaching – Modeling, practice, feedback

Building Engaging Interactions & Environments

Coaching – Focus on new strategies

Coaching

12 Weeks

Creating Opportunities

Prompting & Rewards

Coaching Coaching

Generalization to Daily Activities

Engaging Parents

Week 1 2 3 4 5 6 7 8 11

Coaching

12

Ongoing Support – 3 months

Initial Intro to RP

RP Group

RP Group

Session1 2 3 Monthly-ish Meetings

Reflective Practice

Twice per month feedback (live or based on videos, group or individual)

Ongoing Feedback

Practice with your own clients

Participants

MexicanAmericanFilipino/aAmericanCaucasianMixed

AsianAmerican

Caucasian

Mixed

NotStated

MexicanAmerican

AgencyTrainers

Race/Ethnicity

Therapists

Discipline

21%Hispanic/Latino

35%Hispanic/Latino

Experience

Area AgencyTrainers(n=14)

Therapists(n=43)

SPEDCredential 29% 35%

EIExperience 15.7years (11.35) 10.1years(9.4)

ASDExperience 15.1years(10.22) 8.88years(8.5)

ParentEdExperience 14years (12.56) 7.63years(4.1)

Range1-35yearsforallcategories

Fidelity Data

0

1

2

3

4

5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

0

1

2

3

4

5

0

1

2

3

4

5

AverageNum

bero

fAreas‘Passed’

GroupC

GroupA

GroupB

Fidelity Data

0

0.5

0

0.5

0

0.5

Percen

tageofO

bservatio

nsatF

idelity

Baseline Training

GroupC

GroupA

GroupB

Two Recent Studies

TrainTheTrainer–

CommunityCapacity

PilotStudy–Child&Family

Outcomes

• Study Design– Community trial of Project ImPACT for Toddlers, N=31– Assignment to agency by Regional Center

• Trained therapists• Usual care therapists

– Pre and post assessment by research team

Pilot Study Design

• Child Measures– Characterization:

• ADOS-T• Mullen Scales of Early Leaning

– Pre – Post:• CSBS IT Checklist• Vineland (Parent)• MB-CDI (Parent)• Social Communication Checklist (Therapist)

Pilot Study Design

UsualCare(n=16) ImPACT (n=15)

Age 22.94mo(SD=6.21) 21.47mo(SD=4.73)

Gender 7males(43.8%) 13males(86.7%)

Ethnicity 7Hispanic/Latino (43.8%) 7Hispanic/Latino (46.7%)

Languages

Spanish 5(31.4%) 6(40%)

ADOS-T TotalScore* 9.94 15.80

Pilot Study Participants

CSBS– ByGroup

0

2

4

6

8

10

12

14

16

Intake EndofTreatment Follow-up

SymbolicComposite

UsualCare ImPACT

*

CSBS– ByGroup

0

5

10

15

20

25

Intake EndofTreatment Follow-up

SocialComposite

UsualCare ImPACT

+

ParentCoachingApproach

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

1 2 3 4 5 6

Percen

tageofS

essio

nswith

Paren

tPresent

TypeofParentInvolvementinSession

UsualCare

PIforT

0123456

Score

Code

TrainedTherapists

UsualCareTherapists

ParentEngagementStrateiges

Sustainment

0

10

20

30

40

50

60

70

2016 2017

012345678910

2016 2017

Agencies Children

Thank you!srieth@mail.sdsu.edu

ResearchsupportedbyInstituteofEducationSciences,AutismSpeaks,andtheNationalInstituteofMentalHealth

ASAP Intervention: Outcomes and Methodological Challenges

Brian Boyd1 and Linda Watson2, Michael Alessandri3, Grace Baranek4, Betsy Crais2, Amy Donaldson4, LeAnne Johnson6, Anibal Gutierrez3, Stephanie Reszka2

1University of Kansas, 2UNC-Chapel Hill, 3University of Miami, 4University of Southern California, 5Portland State University, 6University of Minnesota

Overview of ASAP Intervention• Advancing Social-communication And Play (ASAP) is a

manualized, classroom-based intervention for preschoolers with ASD

• ASAP was designed as a supplemental intervention to improve children’s social-communication and play skills

• Developed based on the work on Kasari et al. (2006) & through a prior IES Goal 2 grant (PI: L. Watson)

Kasari, C., Freeman, S., & Paparella, T. (2006). Joint attention and symbolic play in young children with autism: A randomized controlled intervention study. JCPP, 47, 611-620.

ASAP Intervention Components

ASAP Intervention

Team-based

Coaching

Group instruction

Focus on targeted goals

1:1 instruction

Assessment & Monitoring

ASAP Intervention ContentSocial-communication Goals

• Social interaction• Requesting• Joint attention• 20 sequenced objectives

Play Goals

• Exploratory• Relational• Functional• Symbolic• 21 sequenced objectives

Goal MasteryASAP defines mastery as:

“the child showing at least 3 spontaneous occurrences of the targeted skill”

Criteria:• All 3 spontaneous occurrences on same day • Occurs across different situations and varied activities

• During and/or outside of planned, instructional activities

CURRENT ASAP STUDY

Study Design• Multisite, cluster randomized trial with randomization at classroom level• Equal allocation to ASAP or “business-as-usual” control group• Recruited 78 (of 80) classrooms, averaging 2 preschoolers with ASD per

classroom• Classroom teams participated for 1 school year

CONSORT DiagramClassrooms

randomized (n = 78)

Children enrolled (n=161)Intervention classrooms

(n = 40)

Classrooms remained in the study (n = 39)

Children in intervention classrooms (n=85)

Children completing the intervention (n = 82)

Control classrooms (n = 38)

Classrooms remained in the study (n = 36)

Children in control classrooms (n=76)

Children remaining in the control group (n = 73)

ASAP OUTCOMES

Proximal Child OutcomesSocial-Communication (S-C)

• Developed a direct observational coding system

• S-C is coded based on examiner-child interactions during ADOS administration (3x/year)

• Generates scores for social interaction, requesting, joint attention & total S-C

Play

• Adapted a direct observational coding system

• Play is coded based on examiner-child interactions during SPA administration (3x/year)

• Generates scores for exploratory, relational, functional, symbolic & total play

Ungerer, J.A. & Sigman, M. (1981). Symbolic play and language comprehension in autistic children. AACP, 20, 318-337.

Proximal Outcomes• No significant group x time interactions

• No evidence of change over time for either group

Secondary Child Outcomes

• Coding scheme based on Adamson, Bakeman & colleagues

• Data collected at beginning and end of school year in child’s classroom

• 3 five min. observations collected per child (total of 15 min. at each time point)

Adamson, L., Bakeman, R., Russell, C., Deckner, D., (2000). Coding symbol-infused engagement states technical report 9. Atlanta, GA: Developmental Laboratory Dept of Psychology, GSU

Engagement

UnengagementSignificant group x time interactions in favor of ASAP group

d = -.56

Some EngagementSignificant group x time interactions in favor of ASAP group

d = -.50

Methodological Considerations

• Engagement data were collected in children’s classrooms during their normal routines by blind observers

• Social-communication and play data were collected outside of child’s classrooms and semi-structured assessments were administered by naïve examiners

Context-bound

Highly Generalized

ASAP FIDELITY

Teacher Interview

0.00

2.00

4.00

6.00

8.00

10.00

12.00

T1 T2 T3

Adherence

Control Intervention

Methodological Considerations• Measuring a decision-making process

• Sampling across multiple providers for team-implemented interventions

• Determining level of fidelity needed to achieve effects on proximal outcomes

Summary• Found significant differences for secondary but not primary outcomes

• Raises questions about generalization of findings beyond classroom context

• Captured self-reported teacher adherence and differentiated ASAP from BAU classrooms• Measuring how well teachers and teams make data-informed decisions is still a work in

progress• 39 intervention classrooms reflect at least 117 ASAP team members who implemented the

intervention

AcknowledgementsCo-PIs and Investigators

• Linda Watson • Michael Alessandri• Grace Baranek• Betsy Crais• Amy Donaldson• LeAnne Johnson• Anibal Gutierrez

Coaches & Project Coordinators

• Project Coordinators• Stephanie Reszka, Meg Parlade,

LeAnne Hidde, Hannah Sanford

• Coaches• Jen Neitzel, Jessica Dykstra-

Steinbrenner, Kaitlyn Wilson, Emily Monn, Donna Barrow, Cecilia Alvarez-Tabio

AcknowledgmentsProject Staff

• UNC-CH: Jessica Amsbary, Cory Clark, Jenni Reiff, Chetna Sethi, Katie Belardi, Caroline McCarty, Sarah Griffin, Jessica Pulliam, Sallie Nowell, Amanda Plummer, Maggie Fitch, Diana Zellner, Kristin Hodgson, Doug DeMeyer

• UM: Christine Ghilain, Hoa Lam Schneider, Jessica Weber

• UMN: Kelsey Young, Julianne Bazyk, Hannah Vukelich

• PSU: Heather Demosthenes, Haylee Jenkins

Schools & Families

• Thanks to all of the school personnel and families of children with ASD who participated

Funding Source

The research reported here was supported by the Institute of Education Sciences, U.S.DOE, through grant R324A110256.

ExaminingtheEfficacyofClassroomPivotalResponseTeaching

JessicaSuhrheinrich,SarahR.Rieth,&Aubyn C.Stahmer

U.S.DepartmentofEducationGrants:R324A130349,R324B07002

HowcanwemakePRTcomponentsworkforteachers?

Takingturnsandmanagingmaterialswithagroupof

children

Usingpreferredmaterialswithagroup

ofchildren

MakingPRTworkwithothermandatedcurricula/IEPgoals

Providingdirectreinforcementduring

academictasks

Trainingparaprofessionals

Collectingdata

SystematicAdaptationofPRTfortheClassroom

FocusGroups

DevelopManualizedProgram

TestPossibleChanges

Adaptations• Usingcomponentsingroups• Facilitationofturntaking• Usingpreferredmaterialsinacademictasks• UsingmultipleexemplarsResources• Lessonplanning• ExamplesacrossIEPgoaldomains

• Communication;Social;Academic;Behavior

• Datacollection• Paraprofessionaltraining

SystematicAdaptationofPRTfortheClassroom

AdvisoryBoardFeedbackateachStage

FocusGroups

PilotTestAdaptedModel

TestPossibleChanges

(Stahmer,Suhrheinrich,&Rieth,2016)

RandomizedControlledEfficacyTrial

35SchoolDistricts

21EligibleDistricts

19 DistrictsAgreedto

Participate(90%)

17Districtswithparticipatingteachers

2Districtswithnoenrolledteachers

2DistrictsNoResponse

14Districts<15ASD

17schooldistricts

109teachers

76paraprofessionals

257students

94%Female9.2(r=1-34)yearsofexperience

83%Male5.8(r=3-11)yearsold• Autism(92%)• AutismSpectrum(7%)• Non-Spectrum(1%)

SchoolYear

2012/2013

2013/2014

2014/2015

2015/2016

RandomizedWaitlistControlDesign

SchoolYearClassof2013(35teachers)

2012/2013 TrainingYear

2013/2014 Follow-upYear

2014/2015 -----

2015/2016 -----

RandomizedWaitlistControlDesign

SchoolYearClassof2013(35teachers)

Classof2014(30teachers)

2012/2013 TrainingYear ObservationYear

2013/2014 Follow-upYear Training Year

2014/2015 ----- Follow-upYear

2015/2016 ----- -----

RandomizedWaitlistControlDesign

SchoolYearClassof2013(35teachers)

Classof2014(30teachers)

Classof2015(29teachers)

2012/2013 TrainingYear ObservationYear -----

2013/2014 Follow-upYear Training Year ObservationYear

2014/2015 ----- Follow-upYear Training Year

2015/2016 ----- ----- Follow-upYear

RandomizedWaitlistControlDesign

AutismOnly

Cross-Cat

Inclusion

Mild/Mod

Mod/Severe

Resource

ClassroomType

TrainingProtocolTrainingOthers

IntegratingCPRTintoyourclassroom

GroupInstructionwithCPRT

UsingCPRTwithIndividualStudents

CPRTPrinciplesandComponents

AppliedBehaviorAnalysis

• 12hrs grouptraining• Withindistrict• Integratedactivities

• Targetstudents• Focalactivities

CoachingProtocolProcedures:1)Observe;AssessFidelityofImplementation2)Provideverbalandwrittenfeedback,discuss,answerquestions

• Teachers 7.6(1.2)• Paraprofessionals 5.5(1.1)

AverageNumberofSessions

TeacherOutcomes

• HigherfidelityofCPRTforteacherswhocompletedtraining(B=0.27,p=.001).

• TeachersreportusingCPRTfor50min/day,4days/week

TeacherFidelity

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%TrainingYear

Percen

tofp

articipantsre

aching

masterycriteria

TeacherSatisfaction

• 100%ofteachersweresatisfiedorverysatisfiedwiththequalityoftrainingreceived.

• 89%ofteachersreportedCPRTassuccessfulorverysuccessfulforstudentswithASD.

TeacherFidelityatFollow-up

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

TrainingYear

Follow-upYear

Percen

tofp

articipantsre

aching

masterycriteria

TeacherReportedSustainment

• Sustainmentsurveycompleted18moposttraining

Survey Questions Agree orStronglyAgree

Overall, IcontinuetouseCPRT 93%

IammotivatedtouseCPRT. 86%

IusethefullCPRTprotocol withnewstudentswhodidn’tparticipateintheoriginaltraining.

53%

IusepartsoftheCPRTprotocolorcertainCPRTstrategieswithnewstudentswhodidn’tparticipateintheoriginaltraining.

90%

StudentOutcomes

• GoalAttainmentScaling• Socialgoals(t(64)=3.60,p<.001)

• ThePervasiveDevelopmentalDisordersBehaviorInventory(PDDBI)• Sensory/perceptualapproachbehaviors(p=.004)• Repetitive/pragmaticproblems(trend,p=.07)

Stilltoanalyze…

• Childengagementdata• Potentialmoderators

• Teacherfidelity• Studentgains

Discussion

• TeachersupportforCPRT

• Continuedexplorationoffactorsrelatedtoteacherfidelityanduse

• Howcanwemoreeffectivelycapturestudentoutcomes?OR,whyarestudentoutcomessolimited?

• Whatadditionalsupportsareneededforsustainmentofpractice?

Thankyou!

jsuhrheinrich@sdsu.edusrieth@sdsu.edu

astahmer@ucdavis.edu