ORGANIZINGANDUSINGDATAFORDECISIONMAKINGATTIERIThevalueofdataemergesonlywhenanalysisprovidesinsightthatdirectsdecisionsforstudents.

—StephenH.White,BeyondtheNumbers,2005

OrganizingandUsingDataforDecisionMakingatTierI

2nd AnnualIdahoPositiveBehaviorNetworkConference“CreatingSafeandEffectiveSchools”

NadiaK.Sampson, PBISApplicationsTrainingTeamCoordinator

BertEliason,PBISApplicationsTrainingTeam

Tertiary—intensive, individualized5% of student population

Secondary—targeted, small group15% of student population

Universal—primary prevention provided to all students, effective for approximately 80%

ContinuumofDecisionMaking

Allspecialized interventionsaremoreeffectiveandmoredurablewithuniversal,school-widebehavioralexpectationsasafoundation.

Student-CenteredDecisionMaking

EssentialQuestion:Isthestudentsuccessfulatthislevelofsupport?

Studentsarenotlabeledasgreen,yellow,orred;studentsaresupportedwiththeappropriate levelofsupports forsuccess.

Asintensityofsupportincreases, sodoesthefrequencyandintensityofprogressmonitoring.

Math

Reading

Social-Emotional

Writing

Outcome:Enhancedsocial

competence&academicachievement

DataDecisionMakingSupports

SystemsStaffBehaviorSupports

PracticesStudentBehaviorSupports

ResearchonPBISSustainability• Schoolsreportadministrativesupport,staffbuy-in,fidelity,anddatatobeenablers ofPBISimplementation.

• Commonbarriers toPBISsustainabilityareresources,turn-over,fidelity,andstaffbuy-in.

ThefrequencythatdataarepresentedtoallschoolstaffisthesinglefactormostrelatedtohighPBISsustainability.

McIntosh,K.,Kim,J.R.,Pinkelman,S.,Rasplica,C.,Berg,T.&Strickland-Cohen,M.K.(underreview).McIntosh,K.,Predy,L.,Upreti,G.,Hume,A.E.&Mathews,S.(2014).

Fidelity Outcomes

Typesofdata

AdultBehaviorsLeadtoStudentChange

Fidelity Outcomes

• AmajorfeatureofSWPBISisthecommitmenttoongoingassessmentofimplementationfidelity.

MonitoringImplementationFidelity

Overview

• PBISAssessment providessurveys forPBIS

teamstotakeastheyexaminetheirlevelof

SWPBISimplementationfidelity andlookfor

waystoimprovesystemsandpracticesto

benefitstudents,theirfamilies,andtheoverall

schoolculture.

PBISAssessment• Surveysarecompletedonlinewithreportsimmediately availableassoonasasurveyissubmitted.

• PBISAssessmentisfree andrequiresonlythatsomeoneinthedistrictattendawebinartolearnhowtocoordinatethesurveys.

AvailableAssessmentsAnnualAssessment Tool ProgressMonitoringTool

AllTiers TieredFidelity Inventory(TFI)

Tier1(Universal/Primary)

• TieredFidelity Inventory(TFI)• BenchmarksofQuality(BoQ)• Self-AssessmentSurvey(SAS)• EarlyChildhoodBenchmarks

ofQuality(ECBoQ)

• TeamImplementationChecklist(TIC)

Tier2(Targeted/Secondary)&Tier 3(Intensive/Tertiary)

• TieredFidelity Inventory(TFI)• BenchmarksforAdvanced

Tiers(BAT)

• MonitoringAdvancedTiersTool(MATT)

OutcomeTool:SchoolSafety Survey(SSS)&SchoolClimateSurveys*

*Tobereleased

ConnectingFidelity&OutcomesLucky Sustaining

Positiveoutcomes, lowunderstandingofhowtheywereachieved

Replicationofsuccessisunlikely

Positiveoutcomes,highunderstandingofhowtheywereachieved

Replicationofsuccesslikely

LosingGround LearningUndesired outcomes, lowunderstandingofhowtheywereachieved

Replicationoffailurelikely

Undesired outcomes,highunderstandingofhowtheywereachieved

Replication ofmistakesunlikely

Fidelity

Outcomes

WhyDoWeNeedData?Goal

Makeschoolsmoreeffectivelearningenvironments

StrategyRepeatedlygivepeopletherightinformation,attherighttime,intherightformat,whichisthesinglemosteffectivewaytoimprovedecisionmakingandachievevaluedoutcomes

WhyBehavior?Socialbehaviorcontinuestobethesinglemostcommonreasonwhystudentsareexcludedfromschoolsorinstruction.

PerformanceGap&CauseAnalysis

Information

Resources

IncentivesMotives

CapacityKnowledge

PerformanceGap=Thedifferencebetweenwhereanorganizationisandwheretheywanttobe.

PerformanceGap&CauseAnalysis1.Information

2.Resources

3.Incentives4.Motives

5.Capacity6.Knowledge

Environment/System

1—Information• Clearexpectations• Timely,specific

feedback

2—Resources• Materials, tools• Time• Processes

3—Incentives• Financial &

non-financialencouragement

Individual/Person

6—Knowledge• Requisite knowledge

andskillbase

5—Capacity• Abilitytolearn

anddo

4—Motives• Desiretowork

andexcel

Data-BasedDecisionMaking

Decisionsaremorelikelytobeeffective andefficient whentheyarebasedupondata.

Thequalityofdecisionmakingdependsmostonthefirststep– definingproblemstobesolved.

oPrecise(who,what,where,when,howoften,why)

oClear(generalagreementacrossteam)

ImprovingDecisionMaking

PreciseProblemStatement

SetGoals&ProblemSolve

Solution ActionPlanning

PrimaryProblemStatement Solution

ContinuousQualityImprovementIdentifycurrentstatusandproblemswithprecision

Establishgoal(s)

Developsolution(s)Implement

solution(s)withintegrity

Monitoroutcomes&comparetogoal(s)

Reassessandrevisesolution(s)asneeded

ProblemSolvingwithPrecision• Thestatementofaproblemisimportantforteam-basedproblemsolving.• Everyonemustbeworkingonthesameproblemwiththesameassumptions.

• Problemsareoftenframedinthe“primary”form.• Raisesawareness• Notusefulforproblemsolving

• Preciseproblemstatementsresultfromadetaileddatareviewandaresolvable.

ProblemSolvingwithPrecision

PrimaryStatementsy Therearetoomany

referrals

y Gangbehaviorisincreasing

y Thecafeteriaisoutofcontrol

y Studentdisrespectisabigproblem

PrecisionStatementy TherearemoreODRsfor

aggressionontheplaygroundthanlastmonth.Thesearemostlikelytooccurduringfirstrecess,withalargenumberofstudents,andtheaggressionisrelatedtogettingaccesstothenewplaygroundequipment.

BUILDINGDECISIONSYSTEMSMoreThanADataSystem

BuildingDecisionSystems

Guideschoolstobuildcomprehensive“decisionsystems”notjust“datasystems.”

DataSystem DecisionSystemTechnology thathousesdataoDistrictstudentinformationsystem(SIS)

oExcelspreadsheetoSWISapplication

Agreements onthecollectionanduseofdata(includingadatasystem)

RoutinesandprocedurestoembedSWISapplicationdataintodecisionmaking

BuildingDecisionSystems

Analyzingdatainlayersto“drilldown”• Whatisourcurrentreality?• Isthereaproblem?(redflag)

Note:It’soktobedoingwell!• Puttheproblemintocontext (getspecific).

what,where,when,who,howoften,why• Beefficient;don’tdrowninthedata.

DataDecision-MakingSupports

SystemsStaffBehaviorSupports

PracticesStudentBehaviorSupports

Outcomes=

Enhancedsocial

competence&academicachievement

WhatisSWIS?TheSchool-WideInformationSystem(SWIS)isaweb-baseddecisionsystemusedtoimprovebehaviorsupportineducationbyprovidingstaffwithaccurate,timely,andpractical informationformakingdecisionsabouttheschoolenvironment/climate.

BigIdeasMakebetterdecisionsbasedondata• Respondconsistentlytoproblembehavior• Regularlymonitorpatternsofproblembehavioracrosstheschool

• Identifycontexts(e.g.,locations,gradelevels,schedules)wheremoresupportisneeded

• Understandwhyproblembehaviorscontinue(i.e.,perceivedmotivationorfunctionofthebehavior)

• Reducereferraldisproportionalitybyrace,ethnicity,disability,gender,andothercharacteristics

School-wideBehaviorData• CriticalQuestions• Howoftenareproblembehaviorsoccurring?

• When areproblembehaviorsfrequentlyoccurring?

• Where areproblembehaviorsfrequentlyoccurring?

• What problembehaviorsarefrequentlyoccurring?

• Who isfrequentlyengaginginproblembehaviors?

School-wideBehaviorDataSeptember =rateof3.50averagereferrals/day/monthOctober=rateof4.53averagereferrals/day/month

Spikesat9:45AMand1:00PM– 2:30PM

Top3non-classroomlocations =bathroom,playground,&hallway

TuesdayandWednesdayaretheschooldayswiththehighestfrequency.

Top3ProblemBehaviors=inappropriatelanguage,defiance,disruption

5th &8th gradeshavethehighest frequency

25students havemorethan1referral

School-wideBehaviorData

UsingSWISDatatoSolveProblems

Problem

Thefirststepinanyproblem-solving

processisidentifyingthe

problem.

DataAnalysisforPrecision

Location

ProblemBehavior

TimeofDay

PersonsInvolved

Motivation

PreciseProblemStatement

Problemidentificationinvolvesidentifyingthecontext!

UsingSWISDatatoSolveProblems

Knowingthecontextofaproblemhelpsidentifyrelevantsolutions!

Eachteamwillhaveauniqueprocessfordrillingdown

UsingSWISDatatoSolveProblems

Systemvs.SmallGroup/Individual

System-LevelProblemy Thecontextissupporting

theproblem

y Atleast10studentsareinvolved

y Atleast10referrals

y Atleast2staffmembersagree

SmallGroup/Individual Problem

y Wehaveaspecificgrouporstudentthatneedsmoreinstructionorsupport

y Thecontextissupportingmoststudents,sosystemsandpracticesareeffective

It’sokaytobedoingwellandmovetoanotherredflag!

Summary…Problem-Identification• Problemidentificationinvolvesidentifyingcurrentredflagsandtheircontext.

• Preciseproblemidentificationensuresthatproblemsarecurrentandrelevant!

• Knowingthecontextofaproblemhelpsidentifyrelevantsolutions.

• Eachdataanalystandteamwillhaveauniqueprocessfordrillingdown.

Precise(Solvable)ProblemStatementsQuestion Description

q What Whatproblembehavior isoccurring?

q Where Aretherelocationswheretheproblem ismorelikely tooccur?

q Who Howmanystudentsandwhichsub-groupsaremorelikely toengageintheproblem?

q When Aretheretimesofdayordaysofweekwhen theproblem ismorelikely tooccur?

q Why Inthecontextabove,arestudentstryingtoaccessoravoidsomething? Isitaboutattention,tasks/activities, orresources(items)?

ProblemSolvingwithPrecision• TherearemoreODRsforaggressionontheplaygroundthanlastmonth.Thesearemostlikelytooccurduringfirstrecess,withalargenumberofstudents,andtheaggressionisrelatedtogettingaccesstothenewplaygroundequipment.

What? Where? When? Who? Why?

Aggression Playground 1st Recess Largenumberofstudents

To getnewplaygroundequipment

• Notalldatasourceswillhaveefficienttoolsfordrillingdown.

• IsthereanexporttoolthatallowsustopulldataoutandthenaggregateinExceloranotherspreadsheettool?

• Dowehavesomeonewiththeexpertise(e.g.,Excel)andtimetohelpus?

• Whatotherwayscanwetakethe“DrillDown”conceptandfocusconversationsandquestionmoreefficiently?

WhatIfIDon’tHaveaWaytoDrillDown?

GoalSetting&SolutionDevelopmentEssential Elements Explanation

1.PreventionHowcanwe avoidtheproblemcontext?• Who?What?When?Where?

2.TeachingHowcanwe define,teach,andmonitorwhatwewant?• Teachappropriatebehavior,useproblembehaviorasthe

non-example

3.RecognitionHowcanwebuildinsystematic acknowledgment/rewards forpositive behavior?

4.ExtinctionHowcanwepreventtheproblembehaviorfromcontinuingtopayoff?(tiedtomotivation/function ofbehavior)

5.ConsequencesWhatareefficient,consistent consequences forproblembehavior?

6.EvaluationHowwill wecollect andusedatatoevaluate ourfidelityandoutcomes?

SolutionDevelopment&ActionPlanning

Remember toincludethepreciseproblemstatement

aswellasastatement ofthe

goal.

ConnectingOutcomes&Fidelity

Lucky Sustaining

Positiveoutcomes, lowunderstanding ofhowtheywereachieved

Replicationofsuccess isunlikely

Positiveoutcomes,highunderstanding ofhowtheywereachieved

Replicationofsuccess likely

LosingGround Learning

Undesired outcomes, lowunderstanding ofhowtheywereachieved

Replicationoffailurelikely

Undesired outcomes,highunderstanding ofhowtheywereachieved

Replication ofmistakesunlikely

Outcomes

Fidelity

DEMONSTRATION

PBISAppsDemonstrationPBISApps.orgResources• Navigation(Applications,Resources, Supports,DemoAccounts)

PBISAssessment• Overview• Demo(coordination, survey set-up,reports)

SWIS• Overview• SWISDemo(DataEntry,Reports,DrillDown)• CICO-SWISDemo(Enrollment, DataEntry,Reports)• ISIS-SWISDemo(StudentFileSet-Up,DataEntry,Reports)

Formoreinformation…• Visitwww.pbisapps.org formoreinformationaboutanyofthePBISapplicationsorrelatedresources.o PBISAssessment

o SWISSuite(SWIS,CICO-SWIS,ISIS-SWIS)

o PBISEvaluation

•• NadiaSampson:nkatul@uoregon.edu• BertEliason:beliason@uoregon.edu• JessicaDaily:jdaily@uoregon.edu• KatieConley:kconley1@uoregon.edu

• PBISApplicationsContact• www.pbisapps• training@pbisapps.org• support@pbisapps.org

ContactInformation