Embed Size (px)
DESCRIPTION
R URAL D ROPOUT P REVENTION P ROJECT Practice in Motion! E ARLY W ARNING S YSTEMS (EWS) Early warning systems rely on readily available data housed at the school to: Predict which students are at risk for dropping out of high school Target resources to support off-track students while they are still in school, before they drop out Examine patterns and identify school climate issues
Citation preview
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
EARLY WARNING SYSTEMS
JENNY SCALASENIOR RESEARCHER
AMERICAN INSTITUTES FOR RESEARCHFEBRUARY 17, 2015
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
OBJECTIVES• Participants will learn:
o What is an early warning systemo What research says about early indicators of high school
dropouto What is an early warning intervention and monitoring
system implementation process
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
EARLY WARNING SYSTEMS (EWS)
Early warning systems rely on readily available data housed at the school to:• Predict which students are at risk for dropping out of high
school
• Target resources to support off-track students while they are still in school, before they drop out
• Examine patterns and identify school climate issues
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
OTHER EARLY WARNING SYSTEMSWhat are examples of early warning systems that are used in non-educational settings?
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
OVERVIEW OF THE EWS RESEARCH BASE
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
INSTITUTE OF EDUCATION SCIENCES-RECOMMENDED DROPOUT PREVENTION PRACTICES• Diagnostic Practices (Early Warning System)
o Data system and useo Screening
• Targeted Interventionso Adult advocateso Academic supportso Social/behavioral supports
• Schoolwide Practiceso Learning environmento Rigorous and relevant instruction
Dynarski et al., 2008
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
KEY EWS INDICATORS• Engagement
o Attendance/absenteeism• Course performance
o Gradeso Number of credits earnedo Grade point average
• Behavioro Suspensionso Discipline referrals
Research from several U.S. school districts provides a strong foundation for defining early warning signs that students might drop out, but local adaptation is key.
Sources: Allensworth & Easton, 2005; 2012; Balfanz, Herzong, MacIver, 2007; Balfanz, et.al., 2011; Jerald, 2006; Heppen & Therriault; 2008
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
Early warning indicators are used only for prediction—they do not cause students to drop out. Rather, they should
be treated as symptoms of the dropout process that is in progress.
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
“HIGH-YIELD” INDICATORS: HIGH SCHOOL ATTENDANCE
0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40+0%
20%
40%
60%
80%
100%
Graduation Rates by Freshman Absences
Days Absent Per Semester
Perc
enta
ge th
at G
radu
ated
in F
our Y
ears
Source: Allensworth & Easton (2007)
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
“HIGH-YIELD” INDICATORS: HIGH SCHOOL COURSE FAILURES
0 1 2 3 4 5 6 7 8 More than 8
0%
20%
40%
60%
80%
100%
Graduation Rates by Freshman Course Failures
Semester Course Failures
Perc
enta
ge th
e Gr
adua
ted
in F
our Y
ears
Source: Allensworth & Easton (2007)
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
“HIGH-YIELD” INDICATORS: HIGH SCHOOL GPA
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5+0%
20%
40%
60%
80%
100%Graduation Rates by Freshman GPA
Freshman GPA
Perc
enta
ge th
e Gr
adua
ted
in F
our Y
ears
Source: Allensworth & Easton (2007)
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!CHICAGO CONSORTIUM OF SCHOOL RESEARCH’S “ON-TRACK” HIGH SCHOOL INDICATOR
Number of Semester
Core Course Failures
# of Credits Accumulated
Freshman Year
Less than 5 5 or more
2 or more courses Off-track Off-track
0 or 1 courses Off-track On-trackStudents are “on-track” if they:1. Have not failed more than one
semester-long core course, AND2. Have accumulated enough credits
for promotion to the 10th grade.
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
“HIGH-YIELD” INDICATORS: ON-TRACK STATUS
On-Track Off-Track0%
20%
40%
60%
80%
100%
Graduation Rates by Freshman On-Track Status
Perc
enta
ge th
at G
radu
ated
in F
our
Year
s
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
MIDDLE GRADES RISK INDICATORS
GraduatedDid Not Graduate
Sixth grade students demonstrating at least one “flag” had only a 10–20% likelihood of graduating from high school in 5 years.
Engagement• 80% or lower attendance rate
Course Performance• Failing math or English
Behavior• Unsatisfactory behavior grade
Source: Balfanz (2009)
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
COMMON EWS INDICATORS, TIME FRAME, AND THRESHOLDS
Indicator Time FrameThresholds
Middle Grades High School
Attendance•First 20 or 30 days•End of each grading period•End of year
Missed 20 percent or more of instructional
time
Missed 10 percent or more of instructional
time
Course Performance
•End of each grading period•End of year
Failure in an English language
arts or mathematics
course
Failure in one or more courses
Earned 2.0 or lower GPA
(on a four-point scale)
Behavior•End of each grading period•End of year
Locally validated thresholds
End-of-Year Indicator •End of year EWS exit indicator or locally validated
indicators of risk
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
• What resonates with you regarding this research overview? • Do you have access to data that could be used as an early
warning system (attendance, course performance, behavior)?• How are you using this data to support dropout prevention
efforts?• What ideas do you have for how you would define the behavior
indicator? What threshold might make sense to start with for behavior?• What additional questions do you have?
DISCUSSION ON YOUR DATA
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
EARLY WARNING INTERVENTION AND MONITORING SYSTEM (EWIMS) IMPLEMENTATION PROCESS
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
EWIMS SEVEN-STEP IMPLEMENTATION PROCESS
STEP 1Establish roles
and responsibilities
STEP 2Use an EWS Tool
STEP 3Review the EWS
data
STEP 4Interpret the
EWS data
STEP 5Assign and
provide interventions
STEP 6Monitor students
STEP 7Evaluate and
refine the EWIMS process
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
HOW DO YOU USE DATA IN YOUR SCHOOL?• How do you use data to inform decisions in your school? Do
you use a framework?• Does your process overlap with any early warning indicators
data?• What are similarities to those frameworks?• What are some of the differences to those frameworks? • How do you ensure student supports match student needs?
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
STEP ONE: ESTABLISH ROLES AND RESPONSIBILITIES• EWS teams need to include individuals who have:
o Authority to make decisionso Knowledge of diverse studentso Expertise to manage and analyze data
• EWS team are required to:o Meet regularly o Communicate EWS risk or dropout prevention issues to
groups/individuals outside of the teamo Solicit feedback from stakeholders (leaders, staff, students, parents)o Monitor students’ progress
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
STEP TWO: USE AN EWS TOOL• Important EWS tool (middle grades and high school)
considerations: o Data must be regularly entered throughout the school yearo At least one individual should be responsible for ensuring the
EWS tool is loaded with the latest datao EWS Team members must be trained to understand the use of
the indicators within the toolo Reports must be used and accessible in order to make
decisions about students’ needso Student progress in interventions must be monitored
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
STEP THREE: REVIEW EWS DATA• EWS indicators are reviewed and monitored to identify students at risk for
dropping out and to understand patterns in student engagement and academic performance
• Questions to ask about EWS data:o Student-level patterns: What do your data tell you about individual
students who are at-risk?o School-level patterns: What do your data tell you about how the school
is doing?• Are students who were flagged from the beginning remaining “off-track”
through the year?• Are students who were flagged at one reporting period back “on-track” at
the next?
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
EXAMPLE 1: STUDENT-LEVEL REPORT
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
24
EXAMPLE 2: SCHOOL-LEVEL REPORT
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
WHOLE GROUP DISCUSSION TIME: KEY QUESTIONS FOR STEPS 1-3• Step 1: Who needs to be
represented on the EWS team and what types of knowledge do team members need to have?• Step 2: How frequently will
the EWS data be monitored? • Step 3: What do you need
to have in place so you are confident that your data are accurate?
STEP 1Establ ish ro les
and responsibi l i t ies
STEP 2Use the EWS
Tool
STEP 3Review the
EWS data
STEP 4Interpret the
EWS data
STEP 5Assign and
provide intervent ions
STEP 6Monitor students
STEP 7Evaluate and
ref ine the EWIMS process
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
STEP FOUR: INTERPRET EWS DATA• The EWS team must look BEYOND the indicators
o Indicators are just observable symptoms, not root causes
o Root causes emerge through examining additional data from a variety of sources beyond the EWS indicators
• Looking at data beyond EWS Indicators can:
o Help identify individual and common needs among groups of students
o Raise new questions and increase understanding of why students’ fall off-track for graduation
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
STEP FOUR, CONTINUED• Understanding characteristics of students at-risk of dropout is
important because:
o Decisions to persist or drop out are affected by multiple contextual factors - family, school, neighborhood, peers
o Personal and school factors contribute to success or failure during the freshman year
o EWS indicators, such as attendance and course performance problems are distinct indicators of risk, but are highly interrelated, and both can signal disengagement
o Student background characteristics are less important in explaining failures than behaviors in the middle grades and in high school
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
STEP FIVE: ASSIGN AND PROVIDE INTERVENTIONS• The EWIMS team matches individual students to specific
interventions after having gathered information about:
o Potential root causes for individual students who are flagged as at risk
o The available academic and behavioral support and dropout prevention programs in the school, district, and community
• A tiered approach can be used to match students to interventions based on their individual needs
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
TIERED APPROACH TO DROPOUT PREVENTION
~15%
~5%
Tier III/Tertiary Specialized individualized systems for students with intensive needs
Tier II/SecondarySupplemental group systems
for students with at-risk response to primary level
Tier I/Primaryschool-wide
instruction forall Students, including
differentiated instruction
~80% Academic Focus Behavior Focus
Students with disabilities
Receive services at all levels, depending on
need
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
WHOLE GROUP DISCUSSION TIME: KEY QUESTIONS FOR STEPS 4-5• Step 4: What additional
data sources should be used when interpreting EWS data?• Step 5: How confident are
you that the interventions are appropriate for your students?• Step 5: Does your school
have tiered interventions?
STEP 1Establ ish ro les
and responsibi l i t ies
STEP 2Use the EWS
Tool
STEP 3Review the
EWS data
STEP 4Interpret the
EWS data
STEP 5Assign and
provide intervent ions
STEP 6Monitor students
STEP 7Evaluate and
ref ine the EWIMS process
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
STEP SIX: MONITOR STUDENTS AND INTERVENTIONS• The EWS team monitors students who are participating in interventions to:
o Make necessary changes by identifying students’ whose needs are not being met, and/or those students who may no longer be struggling
o Identify new interventions that will to meet students’ needso Use data to monitor the effectiveness of interventions offered
• Increase knowledge about the general effectiveness of interventions• Improve the matching of students to interventions
o Communicate with appropriate stakeholders and solicit their involvement in the process (e.g., feeder schools, next grade level)
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
STEP SEVEN: EVALUATE AND REFINE THE EWIMS PROCESS• Refine the EWIMS Implementation Process
o During the school year
o At the end of a school year
• Identify short- and long-term needs and solutions
o Student needs
o School climate
o Organizational needs (school and/or district)
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
WHOLE GROUP DISCUSSION TIME: KEY QUESTIONS FOR STEPS 6-7• Step 6: How are you using
progress monitoring data for EWS indicators?• Step 7: Identify short- and
long-term successes and challenges to using an EWS.
STEP 1Establ ish ro les
and responsibi l i t ies
STEP 2Use the EWS
Tool
STEP 3Review the
EWS data
STEP 4Interpret the
EWS data
STEP 5Assign and
provide intervent ions
STEP 6Monitor students
STEP 7Evaluate and
ref ine the EWIMS process
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
www.earlywarningsystems.org
RURA
L DR
OPOU
T PR
EVEN
TION
PRO
JECT
Practice in Motion!
Allensworth, E., & Easton, J. (2005). The on-track indicator as a predictor of high school graduation. Chicago, IL: Consortium on Chicago School Research. Retrieved from http://ccsr.uchicago.edu/sites/default/files/publications/p78.pdf
Allensworth, E., & Easton, J. (2007). What matters for staying on-track and graduating in Chicago public high schools: A close look at course grades, failures, and attendance in the freshman year. Chicago, IL: Consortium on Chicago School Research. Retrieved from http://ccsr.uchicago.edu/sites/default/files/publications/07%20What%20Matters%20Final.pdf
Dynarski, M., Clarke, L., Cobb, B., Finn, J., Rumberger, R., & Smink, J. (2008). Dropout prevention (NCEE 2008-4025). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/practice_guides/dp_pg_090308.pdf
Heppen, J., & Therriault, S. (2008). Developing early warning systems to identify potential high school dropouts. Washington, DC: National High School Center. Retrieved from http://www.betterhighschools.org/docs/IssueBrief_EarlyWarningSystemsGuide_081408.pdf
REFERENCES
