The VAM Roadshow

Note: This presentation will be provided electronically.

Current Legislative Session• The current Legislative Session may make changes to SB736

and approve DOE provided rules required by SB736• Decisions made will impact some of the information in this

PowerPoint.

Could possibly be affected:•Explanations of cut points (slide 12)•FAQ (21, 26, 28-31)

Will most likely not be affected:•Definitions/Explanations of VAM, Direct, Indirect, characteristics of VAM (4-11, 13-15)•FAQ information (17-20, 21-22, 25, 27)

What will you learn?• Review of the Value Added Model (characteristics, standard error,

confidence interval, rating information)• How to access student VAM information (only for those with direct

VAM)• General FAQs and Info you need to know

– Rubric changes – Assessments being used or not used – Plan for adding assessments (K-3)– VAM is not CAG friendly, but it can be useful information

• What if an evaluation is below Effective?

What is VAM?• It is different than School Grades and the way we’ve always

used FCAT for things such as learning gains• Covariate adjustment model • Uses student-level prior test scores and other measured

characteristics to predict student achievement• Compares a student’s success to other student’s “like” them to

create a predicted score• Comparisons are created based on Value Added Characteristics• Students predicted score is then compared to their actual score• Currently VAM used for FCAT and Alg I

Mount FCAT

Mount FCAT

School grade:Where are

students on the mountain?

Value Added:Where are

students on the mountain in

relation to other similar

students?

Direct Versus Indirect

• Direct state assessment- An FCAT Course and students took FCAT

• Indirect- Not an FCAT Course but there are students who took FCAT

• School FCAT-Reading VAM• District FCAT-Reading VAM

What characteristics are used to determine student comparisons and a student’s predicted score?

Value Added Characteristics

IndicatorAchievement: Prior YearAchievement: Two Prior Years

Characteristics with greatest

ability to predict score

Other predicting

characteristics

Other characteristics Indicator

Number of students in Class 2Traumatic Brain InjuredEnrolled in 6 or more coursesOther Heath ImpairedGifted Student IndicatorAutism Spectrum DisorderDeaf or Hard of HearingEmotional/Behavioral DisabilityEnrolled in 3 or more coursesEnrolled in 5 or more class periodsEnrolled in 5 or more coursesEnrolled in 4 or more class periodsEnrolled in 4 or more coursesEnrolled in 3 or more class periods

Other characteristics Indicator

Homogeneity of Class 6 Prior Year Test ScoresDual-Sensory ImpairedVisually ImpairedNumber of students in Class 4Homogeneity of Class 4 Prior Year Test ScoresNumber of students in Class 3Homogeneity of Class 5 Prior Year Test ScoresNumber of students in Class 6Missing Mobility Data IndicatorEnrolled in 6 or more class periods

Value Added Information Continued

• VAM scores are not singular (example- your score is a 1)

• VAM scores are actually ranges• Initial range• Standard of Error• Confidence Interval

VAM Information

Unsatisfactory Needs Improvement Effective Highly Effective

Highly Effective

Effective

Effective

Needs Improvement

Unsatisfactory

Effective

Effective

.110-.24

Accessing Direct FCAT VAM Information

Do you know your High Score? Low score? Do you know your school’s high score and low score? Did any

part of your range go into the negative?

Scale Score versus Predicted Score

Accessing the Direct FCAT VAM Information

• Go to the AIMS Companion site at https://rttt.bayschools.net• Log in using your usual sign in credentials• Go to My Appraisals and select the + sign to the far left of the

brown box.• If you have a direct VAM, you have a section titled something

like, “VAM Mathematics/Reading.”• Select the + sign. • Down at the bottom there is a link titled, “Student Details”• You will only have this link if you have a DIRECT VAM (you

teach an FCAT course and students took FCAT)

FAQs and other things YOU

need to know!

Important Rubric Adjustment

• During 10/11 the TASC decided to have different expectations for category 1, 2, 3, and 4 teachers.

• This was approved by the TASC, the Board and DOE.• “When you know better, you do better.”• Scenario:

– Category 4 teachers with an HE Instructional Practice and HE IPDP and an E in Student Growth were rated Effective (2.5 points). A Category 3 or 2 teacher would have had a HE rating with the same amount of points.

• Solution: – For the 12/13 school year, all category of teachers will be rated the same.– The point range for HE for everyone is 2.40 and above.

Will we be able to use Discovery Education Scores?

• At this time there is no plan to use Discovery Education scores for Student Growth

• Discovery Education is used to progress monitor student learning and predict student achievement on FCAT

• Cannot mix Progress Monitoring and Evaluative Data• Varying administration at schools impacts data

reliability

When will all courses be given direct assessments?

• Senate Bill 736 states that by 2014-15 school year, all courses offered by the district must have a direct assessment to assess student growth

• Where do these direct assessments come from?– State Assessments: FCAT, End Of Course Assessments, FAA– District Approved Assessments: Brigance, IB, AICE, Dual Enrollment, AP– Approximately 50% of courses remaining not covered.

• DOE provided Item Bank– Margaret Gamble is working with DOE and CFAC (Central Florida Assessment Coalition) to

provide item writers– Concerns

• Due to be complete 2014-15, the first year the performance pay is mandated by law• Who will maintain it, who will pull assessments, who will assess fidelity, who will input?

– Resolution• We will maintain current Student Growth measurements and add state assessments and

corresponding VAM as we have done with Alg I

• Hope: The Legislature will give us more time.

K-3 Specifically• Issues we are grappling with-

– Must include Student Growth measure (SB736)– DEA: Cannot be used (Progress Monitoring)– FAIR: Currently not aligned to Common Core and will not be aligned

to Common Core until 2014 (Would it be evaluative or progress monitoring)

– SAT 10: Not Aligned to Common Core and expensive– Iowa Assessment: Aligned to Common Core and VERY expensive (70K

per year) – Item Bank: Not available until 14/15– Current solution: Maintain School Level VAM decision– The Oversight Committee plans to revisit this very soon.

VAM isn’t CAG Friendly• We do not get a list that neatly describes all of the variables

and how a teacher’s students do against all the students in the state

• We aren’t given a neat matrix that says, “Teacher A- your ESE students did this, your 4th period did this, etc.”

• We only know if your students met their predicted score or not and which students counted into your VAM

• We are asking DOE if they can provide us the kinds of data people want to know so that we can use it to get better

• But right now, we are not getting that information

So how can we use VAM information?

It is useful!

For the 1st time ever, like students are being compared to like students.

A more complete picture of student learning

Bay District Schools*Excludes schools with no school grade

Low GrowthHigh Achievement 3

High GrowthHigh Achievement 14

Low GrowthLow Achievement4

High GrowthLow Achievement8

Value AddedValue Added

Achi

evem

ent

Achi

evem

ent

High Achievement= A-B School GradeLow Achievement- C-D School Grade

High Growth- Highly Effective- Effective VAMLow Growth- Needs Improvement VAM

Tyndall

What if my overall evaluation falls below effective?

• Breathe.• Don’t panic.• Your first few steps should involve checking all of

your data in AIMS. Sometimes there’s a computer glitch or something was overlooked etc.• THE BEST TIME TO DO THIS IS BEFORE YOU

LEAVE FOR SUMMER BREAK. Go through your instructional practice rating and know what it is and know that it is accurate before you go!!!• What do Option B people check? (IPDP only)

Okay, it’s accurate and below effective. Now what happens?

• This will affect your pay. According to Florida Statute, only teachers with an overall summative rating of effective or higher are eligible for either a step increase or the longevity bonus. You will receive a letter from me explaining the change in your pay (or lack of change as the case may be).

And then?

• We are required to remediate teachers with overall ratings of needs improvement and unsatisfactory.• You will receive a self-assessment that you and

your principal will complete together indicating mutually-agreed upon areas of need.• The professional development team and I will

then review those assessment and match needs with existing professional development (at our expense).

What does a plan look like?

• Each plan is organized by the domains and contains the identified needs, and PD required, to address those needs.• Plans include a combination of Beacon online

classes and TDY days (paid for by the district) with the services of Staff Training Specialists who work with teachers on an individual basis.• Where needs are common, we have after-school

workshops to work with small, focus groups.

This is embarrassing.Who will find out?

• Obviously your principal will know as will the Director of HR. The Staff Training Specialists assigned to work with you will also “know” but must keep the information confidential as part of their job descriptions. STSs work with teachers across the district … no one will know why you are working with one other than you have chosen to do so.

• Note: It is not public record for that school year. However, according to SB 736, the 11/12 evaluation is public record at the end of the following year (one year).

• What can be provided is still up for debate and we are working on clarification (just the summative rating OR the summative pieces– IPDP, Instructional and Student Growth)

Want more information?

• VAM presentation– Project 8 link at

http://www.bay.k12.fl.us/Project8/tabid/2335/Default.aspx

– Go to Florida Value Added and Student Growth

• DOE VAM information– http://www.fldoe.org/committees/pdf/sghandout.pdf– http://www.fldoe.org/committees/pdf/PresentationValue-

addedModel.pdf– www.fldoe.org/committees/doc/Value-Added-Model-

White-Paper.doc

Contact Information

• Sharon Michalik– Executive Director for Human Resources– michasj@bay.k12.fl.us

• Dawn Capes– Instructional Specialist for Race to the Top– capesda@bay.k12.fl.us

• Leon Faircloth– Project Manager for LIIS– fairccl@bay.k12.fl.us