View
217
Download
0
Category
Tags:
Preview:
Citation preview
Release of 2011-12 Preliminary Value-
Added Data Webinar
August 13, 2012
Florida Department of Education
Topics to Discuss Review of state-level results of the model Review of data files received on August 6,
2012 Discussion of the data provided for non-
FCAT subject area teachers and guidelines for the use of these data
Question and Answer session
Florida Department of Education 2
Review of State-Level ResultsThe next series of slides will review: The model fit in 2012 compared to 2011 Comparison of the Reading and Mathematics
VAM scores by Grade Impact analyses showing relationship of VAM
score with various classroom characteristics
Florida Department of Education 3
Data analyses presented on slides 4-15 provided by American Institutes for Research (AIR)
Model Fit: Comparison of R-Square Values for Reading
The R-Square values – which are an indicator of the how well the model fits the data – are high and virtually equivalent to the values observed with the 2011 model
The model behaves the same using FCAT 2.0 data, as it did using FCAT data
Model Fit: Comparison of R-Square Values for Math
The R-Square values – which are an indicator of the how well the model fits the data – are high and virtually equivalent to the values observed with the 2011 model
The model behaves the same using FCAT 2.0 data, as it did using FCAT data
Teacher VAM Score by Grade (Reading)
VAM scores by grade show no particular advantage for any given grade
Teacher VAM Score by Grade (Math)
VAM scores by grade show no particular advantage for any given grade
Impact Analyses for VAM Scores
The following slides show the relationship of the teacher VAM score with various classroom characteristics
The observed score correlations for each characteristic are reported after the scatter plots
In all cases, the correlations are negligible indicating no advantages or disadvantages for any group of students
Correlation of Teacher VAM Score and Percent Students with Disabilities
Correlation of Teacher VAM Score and Percent English Language Learners
Correlation of Teacher VAM Score and Percent Economically Disadvantaged
Correlation of Teacher VAM Score and Percent Gifted
Correlation of Teacher VAM Score and Percent Non-White Students
Correlation of Teacher VAM Score and Mean Prior Achievement
Observed Correlations with Teacher VAM Score
Review of Data Files Received on August 6Files for FCAT Subject Area Teachers (Reading and Mathematics)State-level fileDistrict-level filesSchool-level filesTeacher-level filesStudent-level filesStudent-Teacher link files
Florida Department of Education 16
Review of Data Files Received on August 6 State-, district-, and school-level files provide
summary statistics that many districts have used as the basis for their classification decisions
Teacher-level files provide the performance data for reading/language arts and mathematics teachers Teacher_VAM_estimate is the value-added score Teacher_VAM_estimate_SE is the standard error
associated with the value-added score Pct_Meet_Expectations is the percentage of the teacher’s
students who met or exceeded their expectation as determined by the value-added model
Florida Department of Education 17
Review of Data Files Received on August 6 Student-Level files provide data on the students
included in the analysis, including the predicted (_1112_predictedscore) and actual scores (_1112_scalescore)
Student-Teacher Link files provide the students who were attributed to each teacher in the calculation of his/her value-added score This file should be used to ensure that the appropriate
students are linked to the appropriate teachers Only students for which necessary data (e.g., at least two
years of assessment data) are available are included
Florida Department of Education 18
Review of Data Files Received on August 6Aggregate school- and teacher-level files were also providedThese files combine value-added data (which is run at the subject and grade level) across grades, subjects, and years using the common metric of a proportion of an average year’s growth
The values for an average year’s growth were also provided to districts, as well, on August 6
The value-added scores are converted into proportions and then combinedA request has been made to provide an aggregate file for only 2011-12 (i.e., does not combine across years). That file should be delivered to districts early this week
Florida Department of Education 19
Review of Data Files Received on August 6In addition to files relating to FCAT subject area teachers, files were provided attributing the Reading and Mathematics value-added results to non-FCAT subject area teachersTeacher-level files for non-FCAT subject area teachers were providedStudent-Teacher link file for non-FCAT subject area teachers were provided
Florida Department of Education 20
Data for Non-FCAT Subject Area Teachers and Guidelines for UseSection 1012.34(7)(e), Florida Statutes, requires that for classroom teachers of courses for which the district has not implemented appropriate assessments or for which the school district has not adopted an equally appropriate measure of student learning growth, student learning growth must be measured by the growth in learning of the classroom teacher’s students on statewide assessments
Florida Department of Education 21
Data for Non-FCAT Subject Area Teachers and Guidelines for Use To meet this requirement, student level Reading
and Mathematics value-added results were attributed to non-FCAT subject area teachers serving students who took statewide assessments to calculate two measures that can be used within the evaluation of these teachers The percentage of the teacher’s students
meeting/exceeding expectations as determined by the value-added model
The average of the teacher’s students’ residuals (i.e., differences between the students’ actual and expected scores as determined by the value-added model)
Florida Department of Education 22
Data for Non-FCAT Subject Area Teachers and Guidelines for Use The average of student residuals is
conceptually similar to a teacher’s value-added score
It is a measure reflecting, on average, the degree to which teacher’s students exceeded or missed their expected score as determined by the value-added model
This data can be used in a similar fashion as the value-added score to classify teacher performance
Florida Department of Education 23
Data for Non-FCAT Subject Area Teachers and Guidelines for UseSuggested guidelines for the use of the “average of residuals” scoreIf using state- or district-level summary statistics (e.g., statewide mean) as a basis for classification, rely upon statewide (or district-wide) summary statistics for this group of teachers
Statewide and district-wide summary statistics will be provided to districts this week for this group of teachers
Use the standard error, as your local plans outline, to assist in classifying teacher performance
The August 6 files contained the standard deviation of the average of the residuals; the standard error can be calculated, as well, and will be provided to districts in updated data this week
Florida Department of Education 24
Other Topics Courses used in the calculation of the FCAT
Reading and Mathematics value-added model Two elementary reading/language arts courses
that some school districts began reporting in 2011-12 were not included in the preliminary value-added analysis Course 5010045 Course 5010046
These courses will be added during the reanalysis phase
Florida Department of Education 25
Other Topics Courses used in the calculation of the FCAT
Reading and Mathematics value-added model AP and IB language arts and mathematics courses
have not been included in the FCAT value-added analyses since they are courses with their own assessments measuring the college-level/international standards of those courses
There has been a request to add these courses to the analysis in order to produce an FCAT value-added score for AP/IB teachers who teach students in language arts or mathematics taking FCAT
These courses will be added during the reanalysis phase
Florida Department of Education 26
Other Topics Since the predictions derived from value-added
models are based on how students and classrooms perform statewide, given characteristics controlled for in the model, changes to the underlying data submitted through the review period necessitates a re-calculation of the model
Important, this will result in a new set of teacher value-added scores for all, based on the verified data
Given this fact, we advise that districts await final value-added results, following the review period, before finalizing evaluations
Florida Department of Education 27
Wrap-Up Reminder: Data review period is currently
scheduled to close on Thursday, September 6 at 5:00 EDT
Final value-added data should be delivered by the end of September
We will likely extend the October 19 reporting deadline for evaluation results to the Department
Additional files to be delivered this week are: One-year aggregate FCAT VAM file (2011-12) for teachers
and schools Updated non-FCAT teacher file with the standard error
statistic for the “average of residuals” included State- and district-level summary statistics for non-FCAT
teachers on the “average of residuals” measure
Florida Department of Education 28
Contact Information Juan Copa juan.copa@fldoe.org (850) 245-0744
Questions?
Florida Department of Education 29
Recommended