The Power of EVAAS

Making the Most of Your Data To Inform and Improve

Teaching and Learning

Swain County SchoolsMarch 8, 2013

The Power of EVAAS

Today’s PresentersBecky PearsonProfessional Development ConsultantRegion 8becky.pearson@dpi.nc.gov

Joyce GardnerProfessional Development ConsultantRegion 8Joyce.gardner@dpi.nc.gov

Jason RhodesProfessional Development ConsultantRegion 8email@dpi.nc.gov

Agenda • Welcome, Introductions, Agenda Overview

• Pre Assessment

• EVAAS Basics

• Diving into Reports

• Leveraging EVAAS to Change Instruction

3

Outcomes

• Explore and understand the EVAAS philosophy• Understand and use various EVAAS reports • Using report data to drive changes in teaching that

will impact student learning

Resources

Virtual Resources

Data Literacy Modulehttps://center.ncsu.edu/nc

Pre-Assessment

Poll: I am very familiar with the Educator Val...

Poll: I know how to log in to the EVAAS websit...

Poll: I know how to navigate the EVAAS website...

Poll: I understand EVAAS report names

Poll: I know how to use the EVAAS website to g...

Poll: I know how to access EVAAS reports for i...

Poll: I am able to analyze the metrics in EVAA...

Poll: I know how to collect evidence from EVAA...

Poll: I know how to interpret the following re...

Poll: I am able to communicate the findings of...

Poll: I am able to use data analysis to initia...

What is Data Literacy?

The understanding needed to:

• Find

• Evaluate

• Utilize

data to inform instruction.

A Data Literate Person Can…

A data literate person possesses the knowledge to gather, analyze, and graphically convey information to support short and long-term

decision-making.

Table Talk

• What is EVAAS?• How are you currently using

EVAAS? • What benefits/difficulties have

you experienced?• What have you learned?

Benefits and Considerations for Teachers

• Understand academic preparedness of students before they enter the classroom.

• Monitor student progress, ensuring growth opportunities for all students.

• Modify curriculum, student support, and instructional strategies to address the needs of all students.

Professional Development is

the Key• Data Conversations / True PLCs

• Culture of School

• Sensitivity of Data

• Finger Pointing and Blame Game

• Window vs. Mirror

NC Professional Teaching Standards

Standard I: Teachers demonstrate leadership.

Take responsibility for the progress of all students

Use data to organize, plan, and set goals

Use a variety of assessment data throughout the year to evaluate progress

Analyze data

Standard IV: Teachers facilitate learning for their students.

Use data for short and long range planning

Standard V: Teachers are reflective on their practice.

Collect and analyze student performance data to improve effectiveness

Standard 6 for Teachers

Teachers contribute to the academic success of students.

The work of the teacher results in acceptable, measurable progress for students based on established performance expectations using appropriate data to demonstrate growth.

Benefits for Principals• Gain a consolidated view of student progress and

teacher effectiveness, as well as the impact of instruction and performance.

• Bring clarity to strategic planning and function as a catalyst for conversations that must take place to ensure that all students reach their potential.

• Understand and leverage the strengths of effective teachers.

• Use the valuable resource of effective teaching to benefit as many students as possible.

NC Standards for School Executives

Standard 2: Instructional Leadership• Focuses his or her own and others’ attention persistently and

publicly on learning and teaching by initiating and guiding conversations about instruction and student learning that are oriented towards high expectations and concrete goals;

• Creates processes for collecting and using student test data and other formative data from other sources for the improvement of instruction

• Ensures that there is an appropriate and logical alignment between the curriculum of the school and the state’s accountability program

• Creates processes for collecting and using student test data and other formative data from other sources for the improvement of instruction

Standard 8 for School Executives

Academic Achievement LeadershipSchool executives will contribute to the academic success of students. The work of the school executive will result in acceptable, measurable progress for students based on established performance expectations using appropriate data to demonstrate growth.

Changes in Reporting for 2012-13

2011-12 2012-13

Above

Not Detectably Different

Below

Exceeds Expected Growth

Meets Expected Growth

Does Not Meet Expected Growth

Teacher Ratings Categories

Teachers

1 65432Demonstrate Leadership

Establish Environment

KnowContent

Facilitate Learning

Reflect on Practice

Contribute to Academic

Success

5 Rating CategoriesNot Demonstrated

DevelopingProficient

AccomplishedDistinguished

3 Rating CategoriesDoes Not Meet Expected Growth

Meets Expected Growth Exceeds Expected Growth

Exceeds Expected Growth

Meets Expected Growth

Does Not Meet Expected Growth

2012-13

Table Talk

• How do you explain the concept of Achievement vs. Growth?

Student Achievement

End of School Year

Proficient

Student Growth

End of School Year

Proficient

Start of School Year

Not Proficient

Change over

time

Achievement vs. Growth

Student Achievement: Where are we?• Highly correlated with demographic factors

Student Growth: How far have we come?• Highly dependent on what happens as a result

of schooling rather than on demographic factors

The EVAAS Philosophy• All students deserve opportunities to make

appropriate academic progress every year.

• There is no “one size fits all” way of educating students who enter a class at different levels of academic achievement.

The EVAAS Philosophy• Adjustments to instruction should be based

on the students’ academic needs, not on socio-economic factors.

• "What teachers know and can do is the most important influence on what students learn." (National Commission on Teaching and America's Future, 1996)

Achievement and Poverty

How is this fair?

Academic Growth and Poverty

No one is doomed to failure.

39

Copyright © 2010, SAS Institute Inc. All rights reserved.

High-Achieving Students and ProgressAll schools in Tennessee in 2011 - Math students in grades 4 through 8.

Districts, schools, and teachers that serve high achieving students can make excellent progress, just as easily as those that serve low achieving students.

Proficiency vs. Growth

Scenario Proficient Growth

5th grader begins the year reading at a 1st grade level. Ends the year reading at a 4th grade level.

5th grader begins the year reading at a 7th grade level. Ends the year reading at the 7th grade level.

NO

NO

YES

YES

Table Talk• How could you use the concept of

achievement vs. growth when speaking with parents?

• How does the achievement vs. growth conversation guide PLCs?

EVAAS Overview

What is EVAAS?

SAS EVAAS Analyses

Writing

ACTEnd of Course

End of Grade

LOOKING AHEADPlanning for Students’

Needs:Student Projections to

Future TestsLOOKING BACK

Evaluating Schooling Effectiveness:Value Added &

Diagnostic Reports

How can EVAAS help me?

Improve the

Education Program

EVAAS: Looking BackPast Program Effectiveness

Local Knowled

ge & Expertis

e

EVAAS: Looking Ahead

Incoming Student Needs

• Answers the question of how effective a schooling experience is for learners

• Produces reports that– Predict student success

– Show the effects of schooling at particular schools

– Reveal patterns in subgroup performance

Test Your Knowledge of EVAAS Reports

At your tables, you will find copies of a variety of reports available from EVAAS and labels for each report.

Working with your group, match the report label with the appropriate report.

Reflective Assessments

Value-Added Reporting

• Use to evaluate the overall effectiveness of a district on student progress

• Compares each district to the average district in the state for each subject tested in the given year

• Indicates how a district influences student progress in the tested subjects

District Value Added Report

The School Value Added Report compares each school to the average school in the state.

Comparisons are made for each subject tested in the given year and indicate how a school influences student progress in those subjects.

Value-Added Reporting

• If the Mean NCE Gain is greater than or equal to zero, the average student in this school has achieved a year’s worth of academic growth in a year

• If the Mean NCE Gain is less than zero, the average student in this school has achieved less growth than expected

Mean NCE Gain

Value-Added Reporting

The NCE Base is by definition set at 50.0, and it represents the average attainment level of

students in the grade and subject, statewide.

If the school mean is greater, the average student in the school is performing at a higher achievement level than the average student in the state.

Use to identify patterns or trends of progress among

students expected to score at different

achievement levels

District Diagnostic Reports

District Diagnostic Report• This report is intended for diagnostic

purposes only and should not be used for accountability.

What do you see?

Features of the Diagnostic Report• Quintiles

• Green Zero Line

• Previous Cohort(s)

• Confidence Band

• Whiskers

• 2 Standard Errors

Features of the Diagnostic Report

• Clickable Information

• Reference

• Gain

• Standard Error

District Performance Diagnostic Reports• Use to identify patterns or trends or progress

among students predicted to score at different performance levels as determined by their scores on NC tests

• Students assigned to Projected Performance Levels based on their predicted scores

• Shows the number (Nr) and percentage of students in the district that fall into each Projected Performance Level

Interpreting the Pie Chart

Yellow

Green

Light Red

Your Turn to Interpret

Your Turn to Interpret

The Power of Patterns

Diagnostic Reports Looking for Patterns

School DiagnosticShed Pattern

School DiagnosticReverse Shed Pattern

School DiagnosticTent Pattern

School DiagnosticV Pattern

School DiagnosticOpportunity Gap Pattern

What would an ideal pattern on a Diagnostic Report

look like for closing the achievement gap?

Diagnostic Reports – Desirable Pattern

Diagnostic Report Desirable Pattern

Diagnostic Reports – Whiskers

1. Go to the websitewww.ncdpi.sas.com

1. Go to ncdpi.sas.com

2. BOOKMARK IT!

3. Secure & ConvenientOnline Login

Do you see this?

Reality Check Activity

Using reports from your school, choose one grade level and subject area.

Go to the School Diagnostic Report.

Identify the pattern found in that report. What does this data pattern tell you as

teacher? What are your next steps?*Handout: “Interpreting Your School’s Results”

The Power of the HELP Button

Activity

• Refer back to your Diagnostic Reports.

• Using the HELP button, find information about the red lines called whiskers running vertically through the bars on the graph.

• Share with a partner your explanation of the Diagnostic Report “whiskers.”

Overview of School Effects

Overview of School Effects (sample data)

Overview of School Effects (sample data)

Overview of School Effects (sample data)

Overview of School EffectsIt’s Your Turn!• Find the blank table.

Do this by yourself.

• Using your data

• Fill in your table.

Overview of School Effects

What did you find?• Interesting Patterns• Insights• Areas of Concern• Areas of Celebration

Student Pattern Report

Student Pattern Report

Key points to remember:

• The report shows growth for the lowest, middle, and highest

achieving students within the chosen group.

• The report can be used to explore the progress of students with

similar educational opportunities.

• Like all diagnostic reports, this report is for diagnostic purposes only.

• A minimum of 15 students is needed to create a Student Pattern

Report.

Student Pattern Report

Student Pattern Report

Key Questions

Student Pattern Report – Key Questions

Different experience?Different strategies?Different needs?Number of hours?

Student Pattern Report – Key Questions

Different experience?Different strategies?Different needs?Number of hours?

Rerun the report with new criteria.

YES!

Student Pattern Report – Next Steps

16 Students who attended for 40+ hours

All 31 Students in the Program

Less Informed Conclusion: We need to change the selection criteria for this program.

More Informed Conclusion: We need to adjust the recommended hours for participants.

CUSTOM STUDENT REPORT

Custom Student Report HANDOUT

Academic At-Risk Reports

Academic At-Risk Reports

3 Categories

At Risk- at risk for not meeting the annual federal academic indicators Graduation at Risk-reports for students at risk for not making a Level III on EOC subjects required for graduationOther at Risk-reports for students at risk for not making Level III on other EOC subjects

Academic at Risk ReportsBe Proactive

Use these reports for discussing, developing and implementing targeted intervention and support to students who are at risk for not meeting future academic milestones.

99

Making Data Driven Decisions

What Are Projections?

What Are Projections Anyway?

Given a specific set of circumstances…

…what’s the most likely outcome?

What Are Projections Anyway?Given this student’s testing history, across subjects…

…what is the student likely to score on an upcoming test, assuming the student has the average schooling experience?

EVAAS ProjectionsWhat are they based on?

• Expectations based on what we know» About this student and other students who have

already taken this test

» Prior test scores (EOC/EOG), across subjects

» Their scores on the test we’re projecting to

What’s the Value of the Projections?

Projections are NOT about predicting the future.

They ARE about assessing

students’ academicneeds TODAY.

Assessing Students’ Needs• What are this student’s chances for success?

• What goals should we have for this student this year?

• What goals should we have for this student in future years?

What can I do to help this student get there?

Using Projections to Take Action

• Identify students

• Assess the level of risk

• Plan schedules

• Identify high-achievers

• Assess the opportunities

• Inform

Making Data Driven Decisions

Data Mining

Microsoft Word Document

REFLECTION + PROJECTION = TODAY

Student Project Report

Student Project Report

Student Project Report

Thinking of the State Distribution by QUINTILES

QUINTILE 1

QUINTILE 2

QUINTILE 3

QUINTILE 4

QUINTILE 5

Note the Student’s Projected QUINTILE

QUINTILE 2

Past Effectiveness

Reflecting on Past Effectiveness to Plan for Differentiating Student Instruction

Entering Achievement

Past Effectiveness

Reflecting on Past Effectiveness to Plan for Differentiating Student Instruction

QUINTILE 2

Entering Achievement

ACADEMIC PREPAREDNESS REPORT

Academic Preparedness Report

119

Teacher Value-Added Report

Why should you care about your EVAAS Teacher Value Added Report?• Beginning with your 2013 report, it becomes

part of your evaluation. – Standard 6 – Teachers contribute to the academic success of

their students. (Measurable Progress)– Standard 4 – Teachers facilitate learning for their students

• Teachers plan instruction appropriate for their students– Use data for short and long range planning

– Standard 5 – Teachers reflect on their practice.• Teachers analyze student learning.

Why should you care about your EVAAS Teacher Value Added Report?

You care about your students.

Achievement vs. Progress

Student Progress – How far have I come?

• Highly dependent on what happens as a result of schooling rather than on demographic factors.

Achievement vs. Progress• Focus on progress

• Educators can influence this

• Minimum expectation = one year of academic gain

Understanding Value Added• Projection report looks at past testing information

and projects how a student will perform.– Student’s own past performance

– Performance of students who have taken the test previously

• Students must have three prior test scores for something to be included in the teacher’s predictive report.

• Whole cohort of students analyzed.

EVAAS can tell you WHAT happened. It’s up to YOU to determine WHY it happened and what you want to do about it.

Improve the Education Program

EVAASLocal

Knowledge & Expertise

Info about Teacher Reports• State Growth Standard/State Average =

0.0• Standard Error = a measure of

uncertainty– Usually, the more data you have, the smaller

the standard error.• Index = Teacher Estimate divided by

Standard Error

Effectiveness Categories

Effectiveness Level Determination• Exceeds Expected Growth:

– Teachers whose students are making substantially more progress than the state average

– Index is 2 or greater

Effectiveness Level Determination• Meets Expected Growth:

– Teachers whose students are making the same amount of progress as the state average

– Index is equal to or greater than -2 but less than 2

Effectiveness Level Determination• Does Not Meet Expected Growth:

– Teachers whose students are making substantially less progress than the state average

– Index is less than -2

Evaluation Composite

Index: Teacher Estimate Divided by Standard Error

Courses included in calculation

Statewide distribution of teacher status.

Understanding Teacher Value-Added Reports

• Teacher Estimate: How much progress did this teacher’s students make compared to other students across the state?

• Index: Teacher estimate divided by the standard error. Index is the basis by which teachers are assigned to effectiveness levels.

EVAAS Teacher Value Added Report

EVAAS Teacher Value Added Report

Supplemental Information Table

Student Teacher Linkages

EVAAS Student Report

Teacher Diagnostic Report

Making Generalizations

Making Generalizations• What generalizations

can we make?

• What do we not know?

• How do we find out?

EVAAS Teacher Diagnostic Report

School Composites

Using Teacher Reports to Improve Student Progress

1. Identify highly effective teachers2. Identify teachers who need support3. Identify strengths and areas for improvement

of individual teachers4. Identify school-wide strengths and

weaknesses to inform and provide professional development opportunities

5. Facilitate powerful, crucial conversations between teachers and administrators

6. Impact scheduling decisions

*See “Using Teacher Data”

Role Play Activity

PLC Predictions

and Possibilities

Exit Tickets and Feedback

As you reflect on today’s session, use two sticky notes to capture your thoughts on these topics:

1. Greatest Take Away2. Now, I Need…

http://go.ncsu.edu/ncdpi-resa_survey