The Urban Data Study- What have we learned?
Using Interim Assessment Data to Inform Instruction in Urban
Schools: Perspectives from Districts and Teachers
Documenting Current Practice and Defining Key Dimensions
Ann-Marie Faria, Ph.D. & Jessica Heppen, Ph.D.American
Institutes for Research
1Study TeamJessica HeppenAnn-Marie FariaKatherine SawyerKerri
ThomsenMelissa KutnerWehmah JonesAnestine Hector-MasonSuzanne
StachelMonika TownsendAmerican Institutes for Research
Sharon LewisAmanda HorwitzCandace SimonRenata UzzellMichael
CasserlyCouncil of the Great City Schools
Urban Data StudyFocused on the Use of Interim Assessment Data
for Instructional ImprovementTimely and relevant:Increase in use
and investmentData-driven decision makingARRA funds
3Objectives To document current practice across urban districts
in use and availability of data, particularly interim assessment
data.
To generate empirical evidence about the relationships between
data use practices and improvements in student achievement.
Have 2 reports that address these objectives for dissemination.
The second examines the relationships among data use practices and
teachers use of data to inform instruction. (Links to student
achievement will be the focus of the final report.)4Current
StudyDocumenting Current Practices in Urban DistrictsFocus on
general trends as reported by Council member districts, and more
detailed information gleaned from deep dives in four example
districts.
Links among Key DimensionsFocus on factors that predict teachers
data-driven changes in instruction. 5Theory of Action
6Study Design and SamplesDistrict SurveysPrincipal & Teacher
SurveysSummer 2009; Invited all 67 CGCS member districts54 Research
Directors35 Curriculum Coordinators62 districts represented
SITE VISITS4 participating districts56 teachers28 principals40
district level staff
4 participating districts
3 waves of surveys during 2009-2010
Grades 4, 5, 7, and 8
Wave 1:112 Schools 574 Teachers (4th & 5th)
Wave 2:111 Schools 566 Teachers (4th & 5th)
In June 2009, we administered a survey about district interim
assessments, data systems, and data use in member districts of the
Council of the Great City Schools. The survey was directed to
Academic Chiefs/Curriculum Coordinators and Research Directors and
was open through the summer of 2009. The survey was closed in
September 2009, after a total of 35 Curriculum Coordinators (CCs)
and 54 Research Directors (RDs) completed their respective surveys.
The CCs and RDs who responded represent a total of approximately 62
of the 67 districts in the Councils membership.
Sampled elementary and middle schools (except for one district).
Invited all principals of sampled schools. Invited all teachers of
Math and Reading/Language Arts in grades 4, 5, 7, and 8 in sampled
schools.
Focus here is on the first 2 waves of teacher surveys, with the
elementary school (grades 4 and 5) teachers.
7Assessment ContextNearly all districts administer some sort of
interim assessment
Especially at state accountability testing grade levels
Goals are consistent:
To increase accountability for what is taught in the
classroom.
To ensure more consistent monitoring of school and student
progress
To provide teachers and staff with a tool that will help guide
instructional practices in the classroom.
To prepare students for and predict their performance on the
state assessment.
To inform school improvement planning.
Disconnect between district goals and teacher perceptions
Source: District Surveys; Site VisitsSupports for Data UseData
InfrastructureTimeliness of Data Dissemination
Teachers reported that infrastructure does matter
Professional DevelopmentFrequency of training in using interim
assessment data
Teachers expressed the need for more training on data analysis
and specifically on using data to inform instructional practices in
the classroom.Source: District Surveys; Site Visits9Working with
DataCOLLABORATIONATTENTION TO DATADistrict staff work in
collaboration with schools on interim assessment data
School-based collaboration and data meetingsElementary: Grade
level Middle: Subject area
Item analysis is a common approach to analyzing data ContentDid
I teach this?WordingDid my students understand this question?
Source: Site VisitsFor example, in District C, the data support
team conducts the aforementioned data retreats. These retreats are
half-day sessions in which accountability coordinators compile
comprehensive data for a particular school and meet with the
schools team to discuss how to use the data for school improvement
planning. School leaders select the teams, which range from 2 to 40
people and often include school administrators, the instructional
coach, content leaders, and department chairs.
10Instructional ResponsesReview and Re-teachMost common teacher
approach to student level test resultsReview for the entire
class
School Improvement PlanningAcross all 4 districts, interim
assessment data was incorporated into school improvement plans
Professional DevelopmentTargeted Professional Development based
on interim resultsPD based on the content areas that a majority of
students missed Not necessarily PD to support use of data (i.e. how
to review or retrieve data)
Source: Site VisitsBarriers to Data UseGreatest Barriers to Data
Use:
Lack of time to study and think about data, and to collaborate
with others
Curriculum pacing pressures
Source: Teacher and Principal Surveys; Site Visits12Links Among
Key Dimensions
Mathematics Grades 4 and 5
Source: Teacher Surveys
Reading Grades 4 and 5Source: Teacher
SurveysConclusionsContextual conditions and concrete supports for
data use are linked with actually reviewing interim assessment
data
In turn, actually reviewing data is linked with teachers
changing their instruction.
Teachers who engage in more practices related to reviewing data
engage in more collaboration around data.
Attention to data is related to change in teachers
instruction.
Data Infrastructure and Collaboration were not connected to the
data use process as expected.Structured time 16Summary and Next
StepsMost districts conduct interim assessments to gauge student
progress in a consistent way.
Infrastructure to support data use is in place and continuously
improving.
Using data to inform decisions is a stated priority by urban
districts across the nationIn the 4 districts, communication varies
and at the school level there are different data cultures and
perceptions about the importance of data that are related to actual
use.
17Key RecommendationsImprove communication with school-level
staff by clearly articulating the goals of the interim assessment
approach.
Continue to work to garner support from principals and other
school leaders.
Allow enough time to gauge the effectiveness of interim
assessment strategies.
Report interim assessment results as quickly as possible, and
increase teacher access to varied forms and levels of data.
Provide structured time for teachers to review data and consider
how the results should inform their instruction.
Increase opportunities for professional development on how to
use student achievement data to inform instructional decisions.Next
Steps Next steps include:Collecting Spring 2010 student achievement
data (Fall 2010)
Producing future reports that focus on how data use practices
are linked with student achievement
19Thank you!
Contact Information:
Ann-Marie Faria, Ph.D.Research AnalystAmerican Institutes for
Research 1000 Thomas Jefferson Street, NW Washington, DC 20007
Phone: 202-403-5356 Email: [email protected]
