Upload
grace-carpenter
View
212
Download
0
Tags:
Embed Size (px)
Citation preview
Jim Lloyd_2007
Educational Value Added Educational Value Added Assessment System Assessment System
(EVAAS)(EVAAS)Olmsted Falls City Schools Initial Olmsted Falls City Schools Initial
Presentation of 4Presentation of 4thth Grade Grade StudentsStudents
Suppose this box represents Suppose this box represents a group of students for a a group of students for a
given grade level and given grade level and subject area… subject area…
Low
High
High
Low
… … and we fast forward to and we fast forward to the end of the given school the end of the given school
yearyear
An Achievement Status ConsiderationAn Achievement Status Consideration
Considerations for Gauging Effectiveness:
1) External Standard (proficiency level)
2) Student’s Ending Point of Achievement
? (Yes/No)
… sets a proficiency level students are expected to meet.
A Value-Added ConsiderationA Value-Added Consideration
Considerations for Gauging Effectiveness:
1) Student’s Prior Achievement Level
2) Student’s Ending Point of Achievement
… sets one end of the measurement at the students’ entry point, not at an external standard.
How much gain?
Factors related to student learning - Factors related to student learning - District, School, and Teacher Influence District, School, and Teacher Influence
on Student Progresson Student Progress
Following inferences were shared at the Following inferences were shared at the Governors Education Symposium (2004)Governors Education Symposium (2004)
Based on 22 years of Value-Added Study, Dr. Based on 22 years of Value-Added Study, Dr. Sanders draws the following conclusions:Sanders draws the following conclusions:
Variation in student academic progress can be Variation in student academic progress can be attributed this way:attributed this way:
5% attributed to District quality5% attributed to District quality
30% attributed to School quality 30% attributed to School quality
65% attributed to Teachers quality65% attributed to Teachers quality
Difference in student achievement levels after Difference in student achievement levels after comparably matched 2comparably matched 2ndnd graders experience qualitatively graders experience qualitatively
different teacher sequences of effectiveness, TNdifferent teacher sequences of effectiveness, TN
L=Least Effective H=Highly Effective (Sanders & Rivers, 1996)
Difference in student achievement levels after Difference in student achievement levels after Experiencing qualitatively different teacher Experiencing qualitatively different teacher
sequences of effectiveness, Dallassequences of effectiveness, Dallas
Source: Heather Jordan, Robert Mendro, & Dash Weerasinghe, “Teacher Effects On Longitudinal Student Achievement” 1997.
76
27
15
90
Beginning 3rd Grade
Score (Percentile)
Ave
rag
e M
ath
Sco
re 3
yrs
lat
er i
n P
erc
en
tile
s
Dallas StudentsAssigned To 3Highly EffectiveTeachers In ARow
Dallas StudentsAssigned To 3IneffectiveTeachers In ARow
Difference in student achievement levels after Difference in student achievement levels after experiencing qualitatively different levels of experiencing qualitatively different levels of
teacher effectiveness, Bostonteacher effectiveness, Boston
-0.6%
14.6%
0.3%
5.6%
-5
0
5
10
15
Average Student Growth
OverOne Year
(percent)
Least Effective TeachersMost Effective Teachers
Source: Boston Public Schools, in “High School Restructuring,” March 9, 1998.(courtesy of The Education Trust)
READING
MATH
Rivers (1999)Rivers (1999) Can varying sequences of teacher Can varying sequences of teacher
quality vary student chances of passing quality vary student chances of passing a high stakes test? a high stakes test? o Rivers (1999) followed students from 4Rivers (1999) followed students from 4thth through 9 through 9thth grade grade
from two large urban districts. from two large urban districts.
o Students were grouped into quartiles on the basis of Students were grouped into quartiles on the basis of achievement on the 4th grade standardized test.achievement on the 4th grade standardized test.
o Math teachers were identified for grades 5-8 and were Math teachers were identified for grades 5-8 and were linked to effectiveness levels (Low, Avg, High), based on linked to effectiveness levels (Low, Avg, High), based on their success in facilitating value-added achievement with their success in facilitating value-added achievement with a a priorprior group of students. group of students.
o Do these quality distinctions apply to another cohort of Do these quality distinctions apply to another cohort of students and offer reliable information about how a students and offer reliable information about how a student will do on a high stakes in the future?student will do on a high stakes in the future?
Probability of Passing High School Exam based on Sequence
of Teacher Effectiveness Experienced
Jim Lloyd_2006
How is value added different How is value added different than what we’re currently than what we’re currently
doing?doing? Current system measures only Current system measures only achievementachievement (what percentage (what percentage of kids pass the test).of kids pass the test).
New system measures New system measures progressprogress towards the standard, so can more towards the standard, so can more accurately reflect the growth of ALL accurately reflect the growth of ALL students, especially those who may students, especially those who may be hard-pressed to be hard-pressed to everever reach the reach the passing rate. passing rate.
Jim Lloyd_2006
How will this help me as an How will this help me as an educator?educator?
Measures growth from Measures growth from where the student where the student enters the grade level.enters the grade level.
Measures progress at Measures progress at building/grade levels.building/grade levels.
Shows if Shows if programs/instruction programs/instruction are impacting student are impacting student growth.growth.
Aligns classroom Aligns classroom resources to help all.resources to help all.
Jim Lloyd_2006
EVAASEVAAS
How to view a summary How to view a summary reportreport
Jim Lloyd_2006
Concepts to RememberConcepts to Remember This isn’t about proving…it’s about This isn’t about proving…it’s about
improving.improving.
The power of the scores is in the sample The power of the scores is in the sample size. size. The scores become less reliable as the size of The scores become less reliable as the size of
the population decreases.the population decreases.
We will not be looking at the individual We will not be looking at the individual growth of individual students because it is growth of individual students because it is not reliable.not reliable.
Jim Lloyd_2006
Colors to RememberColors to Remember
A score of “0” isn’t bad because we A score of “0” isn’t bad because we have a 0 growth model.have a 0 growth model.
RedRed isn’t good isn’t good YellowYellow is no detectable difference is no detectable difference GreenGreen is good is good
Jim Lloyd_2006
Terms Terms Mean NCE Gain—theMean NCE Gain—the average gain that average gain that
the group achieved compared to where the group achieved compared to where they started. This is the growth they started. This is the growth measure for the entire group of kids measure for the entire group of kids who have at least 2 data points.who have at least 2 data points.
Standard Error—aStandard Error—a statistical calculation statistical calculation as to how confident we are that the as to how confident we are that the score didn’t occur due to error.score didn’t occur due to error. Impacted by sample size and dispersionImpacted by sample size and dispersion Ohio’s model uses 1 SEM for significanceOhio’s model uses 1 SEM for significance
Jim Lloyd_2006
Terms cont.Terms cont.
State Base Year—theState Base Year—the average amount of average amount of growth that happened by the grade in growth that happened by the grade in Ohio (expressed in NCE units)Ohio (expressed in NCE units)
2005 Mean—the2005 Mean—the average amount of average amount of achievement that the achievement that the studentsstudents came in came in with.with.
2006 Mean—the2006 Mean—the average amount of average amount of achievement that the achievement that the studentsstudents left with. left with.
The amount of growth is the difference The amount of growth is the difference between the 2005 and 2006.between the 2005 and 2006.
Types of ReportsTypes of Reports
CompositeComposite Diagnostic Subject Specific Diagnostic Subject Specific
ReadingReading MathMath
Performance Subject SpecificPerformance Subject Specific ReadingReading MathMath
Subgroup specific (reading/math)Subgroup specific (reading/math)Jim Lloyd_2006
Composite ReportsComposite Reports
Jim Lloyd_2006
Jim Lloyd_2006
Composite ReportComposite Report
Shows the total amount of growth by Shows the total amount of growth by combining all achievement tests combining all achievement tests administered.administered.
We’re going to look at our composite We’re going to look at our composite report now.report now.
Jim Lloyd_2006
Building Summary ReportBuilding Summary Report
There are summary reports for each There are summary reports for each building as well that are subject building as well that are subject specific.specific.
Here is reading.Here is reading.
Jim Lloyd_2006
Subject Specific ReportsSubject Specific Reports
Jim Lloyd_2006
Jim Lloyd_2006
Subject Specific ReportsSubject Specific Reports
Shows the amount of growth Shows the amount of growth obtained in a particular subject area.obtained in a particular subject area.
We’re going to look at a Summary We’re going to look at a Summary Reading and Math Report now.Reading and Math Report now.
Jim Lloyd_2006
What is this? Growth standard of 0 is what we would expect if the student made a year’s growth
Jim Lloyd_2006
EVAASEVAAS
How to view a Diagnostic How to view a Diagnostic ReportReport
Jim Lloyd_2006
Pos Gain
Neg Gain
This shows us the # of OFCS students in each quintile
Total # of students in the testing pool is divided into 5 groups called quintiles
A “0 gain model” means that if students made about a year’s worth of growth we would see a gain within +1 or -1 of the standard error
Jim Lloyd_2006
Note that without at least 5 in a group, no data is reported
These students all have plenty of room to grow…even those at the 97th NCE.
Jim Lloyd_2006
Example—Scattered spread w/a low progress spike
Jim Lloyd_2006
Jim Lloyd_2006
EVAAS Math ReportsEVAAS Math Reports
District Reports 2006District Reports 2006
Current 6Current 6thth Grade Students Grade Students
Jim Lloyd_2006
Building Summary ReportBuilding Summary Report
There are summary reports for each There are summary reports for each building as well that are subject building as well that are subject specific.specific.
Here is mathematics.Here is mathematics.
Jim Lloyd_2006
Math Diagnostic ReportsMath Diagnostic Reports
What is the gain pattern?
Remember to look @ GAIN & SEM
Implications of EVAASImplications of EVAASQuestionsQuestions
What sorts of organizational What sorts of organizational strategies will lead to strategies will lead to increasing district, building and increasing district, building and teacher effectiveness?teacher effectiveness?
What are the professional What are the professional development implications of development implications of instituting EVAAS in Olmsted instituting EVAAS in Olmsted Falls?Falls?
What is the short and long What is the short and long term plan to prep the term plan to prep the administrative team and administrative team and building staff for this additional building staff for this additional accountability system?accountability system?
ResponsesResponses Teachers and principals make Teachers and principals make
the difference. Build capacity!the difference. Build capacity! Make a concerted effort to get Make a concerted effort to get
ahead of the curve and use this ahead of the curve and use this and other data to our and other data to our advantage.advantage.
Foster and reinforce a data-Foster and reinforce a data-driven culture across the district.driven culture across the district.
Build upon and expand our Build upon and expand our staffs’ capacity to implement staffs’ capacity to implement and monitor a standards-based and monitor a standards-based educational program for all educational program for all students.students.
Jim Lloyd_2006
Questions?Questions?