Embed Size (px)
Citation preview
Rightsizing Assessment: A Set of Tools and a Process Moving States and Districts Toward High Quality, Coherent, and Meaningful Assessment Systems
June 21, 2016
Session Goals
2
◆Provide an opportunity for participants to reflect on the quantity and
quality of the assessments administered to students in their states
and districts.
◆Understand how an assessment inventory process has helped
districts and schools address overtesting concerns.
◆Reflect on state, district, and RESA approaches to supporting an
inventory process.
◆Understand opportunities through ESSA to support assessment
audits.
April 2008 – Stiggins’ Assessment Manifesto: A Call for the Development of Balanced Assessment Systems
March 2010 – Using Balanced Assessment Systems To Improve Student Learning and School Capacity: An Introduction by Gong
June 2014 - Achieve releases Student Assessment Inventory for School Districts
Rightsizing…
gaining momentum
3
October 2014 – Commitments from CCSSO and CGCS on High-Quality Assessments
◆Assessments should be high quality.
◆Assessments should be part of a coherent system.
◆Assessments should be meaningful.
June 2015 - CCSSO releases Comprehensive Statewide Assessment Systems: A Framework for the Role of the State Education Agency in Improving Quality and Reducing Burden
4
Rightsizing…
gaining momentum
October 2015 – Student Testing in America’s Great City Schools: An Inventory and Preliminary Analysispublished by CGCS
October 2015 – ED releases Testing Action Plan
November 2015 – Fewer and Better Local Assessments: A Toolkit for Educators released by Education First
5
Rightsizing…
gaining momentum
December 2015 – ESSA passed
Section 1202 State Option to Conduct Assessment System Audit
February 2016 – Letter from ED Secretary to Chief State School Officers which included Leveraging ESEA Funds to Support Fewer, Smarter High-Quality Assessments
6
Rightsizing…
gaining momentum
…and encouraged by testing pushback...
Opting out
Concerns expressed by parents, teachers, and school leaders about the amount of testing
7
Rightsizing…
gaining momentum
While parents do have
concerns about testing...
8
In your opinion,is there too much emphasis on standardized testing in the public schools in your community, not enough emphasis on testing, or about the right amount?
Source: Phi Delta Kappa/Gallup. (2015). The 46th Annual PDK/Gallup Poll on of the Public's Attitudes Toward the Public Schools. http://pdkpoll2015.pdkintl.org/wp-content/uploads/2015/10/pdkpoll47_2015.pdf
9
Source: Phi Delta Kappa/Gallup. (2015). The 46th Annual PDK/Gallup Poll on of the Public's Attitudes Toward the Public Schools. http://pdkpoll2015.pdkintl.org/wp-content/uploads/2015/10/pdkpoll47_2015.pdf
How important do you think each of the following is for measuring the effectiveness of public schools in your community?
Scores that students receive on standardized tests
...public school parents do
think standardized testing is
important.
Parents and voters know very little about new state assessments.
Basic information, especially information that explains why the new assessments are good for students, is badly needed.
Tests that are most valued measure real learning, assess problem solving and critical thinking, and determine baselines for knowledge.
However, a plurality of voters and a majority of parents say students are given too many standardized tests in public schools.
Findings from Achieve’s National Poll of Parents and Voters on Assessments
10
What concerns about student assessment are you hearing?
11
Addressing the concerns
12
AssessmentInventory
A School District View
A State Approach A RESA
Approach
Recommendations
The Student Assessment
Inventory
◆ There are legitimate concerns from
parents and the public about too much
testing.
◆ The volume of testing goes well beyond
those required by states, and the layers
of state, district, and school assessment
do not always add up to a cohesive,
aligned, informative whole.
◆ Achieve’s Student Assessment Inventory
for School Districts is designed to assist
district leaders in taking stock of their
assessments and assessment strategy.
Addressing concerns about too much testing
14
◆ The objective is to determine the minimum testing necessary to serve
essential diagnostic, instructional and accountability purposes.
◆ Taking stock and then taking action requires significant district
commitment.
◆ The inventory tool is only one element of a thoughtful longer process that
both engages productively with concerns about testing and leads to real
changes in testing practice.
◆ The inventory is not a one-time event. Districts should regularly re-examine
their assessments in light of changing district needs and improvements in
available assessments.
◆ The inventory and related resources are free to use and adapt.
What is the Student Assessment Inventory for
School Districts?
15
How was the Assessment Inventory developed?
16
Achieve developed the assessment inventory to support a
voluntary, district-led process:
◆ Achieve developed an initial draft of the inventory tool and shared with a
broad network of state and district leaders and experts for feedback.
◆ In partnership with the Connecticut State Department of Education, Achieve
piloted a revised version of the tool with a group of eight districts across
Connecticut. Achieve finalized the tool based on feedback from these
districts.
◆ Achieve continues to work with states, districts and partners to develop and
share additional resources to support use of the inventory.
17
Assessment inventory resources
◆ Training materials
◆ Setting the Context (90 minutes)
◆ Inventory Planning (4 hours)
◆ Stakeholder Mapping
◆ Assessment Inventory Training Scenarios
◆ Communications and Messaging
◆ District Guidance
◆ Sample teacher, parent, and student focus group and survey protocols
◆ Considerations for Special Education Assessment Systems
◆ Considerations for Assessing English Language Learner Students
◆ State adaptations of the inventory – Delaware, Idaho, Illinois
Components of the Assessment
Inventory
Four stages of the process
19
Reflect and Plan
Conduct the Inventory
Analyze the Inventory
Make Recommendations
1
2
3
4
Reflect and Plan: Guiding Questions
20
Districts use a set of guiding questions to initiate the planning process.
◆ What is the district context in which the inventory is being considered?
◆ What are the objectives of the student assessment inventory?
◆ Who will collect the information needed for the inventory table? How will they
access that information?
◆ What is the scope of the inventory? Which assessments should be included
and excluded from the inventory table?
◆ What individual or entity has the authority to act on the results of the
inventory? Who will be making the recommendations?
◆ There are three types of questions being asked in the table:
◆ Basic information questions
◆ Use/purpose questions
◆ Operational questions
◆ Some information to complete the table will not be directly available from test
specifications and will require communicating with users of the assessment,
especially with respect to issues of assessment use. A short survey or set of
focus groups is strongly recommended to better understand how
assessments are being used by multiple audiences.
Conduct the Inventory: Inventory Table
21
Inventory Table: Basic Information Questions
22
Inventory Table: Use/Purpose Questions
23
Inventory Table: Operational Questions
24
In analyzing the inventory, it is critical to do several levels of analysis.
◆ Developing a student-level perspective by looking across all
assessments students take at a particular grade level or grade band, and
then by particular student needs and characteristics.
◆ Identifying assessments that the district will continue to administer,
ensuring a shared understanding of their intended purpose(s) and actual
use(s), and determining if any changes are necessary (e.g. to improve test
quality, alignment or utility).
◆ Identifying the assessments that may be eliminated or where significant
changes are necessary.
◆ Helping districts build toward recommendations while re-engaging with
key stakeholders to review potential options and decision points.
Analyze the Inventory
25
Based on the inventory analysis, what recommendations will the district
make to streamline and/or strengthen its assessment program?
Make Recommendations
26
What do you think is the most difficult part of
the Student Assessment Inventory for school
district staff? Think about the inventory tool
and the process itself.
Question
27
28
Examples of recent momentum
◆ Connecticut provided grants to nearly 50 districts to go through an assessment inventory
process by summer 2015.
◆ Delaware provided grants to all districts to streamline assessments using Achieve’s
assessment inventory process, and has used the tool to inventory all statewide
assessments.
◆ Idaho and Illinois modified and circulated versions of Achieve’s tool. Kentucky shared the
Achieve tool with all districts.
◆ CCSSO guidance to states featuring Achieve’s assessment inventory.
◆ EdWeek article featuring HQAP Illinois districts and ISBE .
◆ High Quality Assessment Project (HQAP) project with 15 districts in four sites.
◆ Achieve partnership with the National School Boards Association to support districts and
focus on role of school boards in supporting high quality assessments.
29
High Quality Assessment Project (HQAP) grant
◆ Four sites selected in January-February 2015 to receive $40,000 grants to
complete the assessment inventory and make recommendations to decision-
making body (e.g., school board, superintendent).
◆ Capitol Region Education Council (CT) (Bloomfield, Branford, East Lyme,
Hartford, Region #6, Tolland, Vernon)
◆ Educational Service Center (ESC) of Central Ohio (Buckeye Valley, Olentangy,
Westerville, Worthington)
◆ Knox County Schools
◆ Illinois (Bensenville, Urbana, West Aurora, ISBE)
◆ Achieve provided ongoing support through April 2015 cross-site convening, July
2015 cross-site webinar, monthly calls with each site, communications support,
and resource development.
Assessment Inventory Results
31
Results
◆ Reduction or elimination of assessments. In two sites, Illinois and Ohio,
reductions in testing were significant, particularly for benchmark/interim and K-2
diagnostic assessments for reading.
◆ Increased engagement with stakeholders. All sites reported benefits from
engaging stakeholders in the process and in particular parents.
◆ Identification of training needs. Several districts identified the need to provide
training to teachers and other staff on the purpose, administration, and retrieval
and analysis of the data of specific assessments. Assessment literacy training was
also identified as a priority and especially how to make informed decisions about
creating and/or purchasing assessments and how to appropriately interpret the
assessment results.
◆ Assessment Inventory in Action brief describes the work of the HQAP districts.
32
Additional Lessons Learned
◆ The testing burden related to special populations is real - and more
nuanced.
◆ It is challenging to make local decisions when the state approach to
summative assessments is in flux.
◆ There are tremendous gaps in assessment literacy amongst stakeholders
within the system. A level-set may be needed.
◆ Conversations may range from technical to more emotional. Stakeholders
have a range of interactions and experiences with assessment.
◆ Schools may interpret voluntary guidance from districts as a stronger
signal than the district intends.
33
Benefits to Advancing this Work
◆ Driving a process to streamline assessment pushes the conversation about quality
and can help create the conditions to spur innovation.
◆ Engaging with stakeholders – including students, parents and educators-
throughout the process can help build the narrative for the story you want to tell.
◆ Evidence and feedback gathered can be used to demand better, higher quality
and more aligned assessments. This is particularly helpful in vendor selection, and
even more powerful when done collectively as a district or group of districts.
◆ This process helps build assessment literacy for all involved – this is good
professional learning.
◆ This process can provide greater transparency and strengthen communication
amongst stakeholders - push on the public reporting and data elements of the
remaining assessments.
The Student Assessment
Inventory: A District View
Expectations
Save time
Save money
Assessment information will be easily accessible
35
Districts:
Expectations Versus Reality
Reality
There was no comprehensive repository for all assessments, costs, and
purpose for all grade levels;
Some district-adopted assessments were not being used;
Some additional assessments were administered without district
knowledge;
Stakeholders were not aware of the purpose for some assessments (and
did not administer of analyze them with fidelity);
Stakeholders gave differing feedback on the importance and utilization of
some assessments.
36
Districts:
Expectations Versus Reality
The Need To:
Provide consistency in test administrative practices across the district;
Enhance focused professional development on assessment administration
and analysis;
37
Lessons Learned from Districts
Communicate, Educate, and Communicate
Parents
Monthly updates regarding assessment – issues/changes
Digital Communication (websites, emails)
Letters, Newsletters, Surveys (results)
District Parent Groups (PTO/PTA) – focus groups / 2-way communication
Staff
Monthly updates regarding assessment – issues/changes
District Assessment Inventory;
Professional Development
Students
Lessons Learned from Districts
38
The Power of Why!Simon Sinek
Clearer communication to stakeholders;
Professional dialogue on purpose of assessments, beliefs, and values (online
assessments, increased rigor);
Reduced frequency of tests in some grade levels due to state testing conflicts;
Reduced redundancy of testing similar outcomes;
Replaced assessments in order to garner more productivity.
District Outcomes
39
Delaware:
A State Approach
Delaware Timeline for Scaling Up
Prior to Winter 2015 State Assessment Inventory
Five Year Assessment Plan
March 2015 Governor’s Announcement
DOE presentation to all district/charter superintendents
April 2015 Grants available
Detailed process available to all districts
Public comment for 3 months
May 2015 Website with resources available to districts
DOE releases State Assessment Inventory as example for districts
41
Delaware Grants for Districts and Charters
DOE released a grant opportunity to fund district ability to: Complete the assessment inventory
Develop an action plan to execute on findings
Develop a communications and community engagement process
Award varied based on district size
Awards could be used for: Full-time staff member or contractor to facilitate process
Teacher/parent stipends
Meeting expenses
Communication Plans
http://www.doe.k12.de.us/Page/2743
State Assessment Inventory
Provided inventory of all current state assessments - mandatory
and optional
Exception: did not include at state level all assessments for teacher
effectiveness
43
Delaware Resources Available
44
Guidelines for the Roll-out - Discussion
Area Guiding Questions
Policy Changes • What policies would we need to adjust or put into place?
Stakeholder
Engagement and
Communication
• Which stakeholders will need to be engaged (outside of this group)
and how will we engage them?
• What communication strategies and feedback loops will help ensure
stakeholders understand and feel connected?
Resources and
Support
• Will this require additional funding? Staffing?
• What tools and resources will be necessary for use in districts and
schools?
• What external organizations might be available to support?
Scope • What activities will districts need to complete (ex. Develop plans to
review existing assessments? Streamline as needed?
Timeline • What is the most appropriate timeline to complete this work?
Professional
Learning
• What professional development is needed for districts and schools?
At what point and through what mechanism?
Delaware Timeline for Scaling Up
July 2015 Legislation creating a Assessment Inventory Committee to review State and
District Assessments and provide recommendations to DOE
September 2015 Assessment Inventory due from districts
December 2015 Final Report with recommendations due from districts
Submit Impact Report of Process (testing time before and after)
Compiled and share with Assessment Inventory Committee
June 2016 Final Report from AI Committee
46
Transparency of information
districts required to submit information about impact
information posted on district and state websites
District Summaries
47
Delaware Lessons Learned
Important to include communications staff early on district buy-in
Can never provide too many resources
LEAs want examples as models
State Assessments will be part of the conversation
Consistency of format is critical if summarizing at the state level
Transparency of all parts of the process districts push that testing burden was from the state
48
A Regional Education
Service Agency (RESA)
Approach
Four school districts in the Columbus area volunteered to pilot the
Assessment Inventory with the Educational Service Center of
Central Ohio (ESCCO) in their districts during Spring 2015.
Drs. Neil Gupta and Cathy Heidelberg co-led the pilot for the
ESCCO.
Ohio’s Pilot - RESA
50
RESA’s provide expertise in project planning with milestones,
timelines, and communication plans;
RESA’s can provide a forum for districts to leverage shared
learning, assessment expertise, and assessment vendor
coordination.
The Need for RESA Involvement
51
Benefits to RESA’s
RESA’s Benefits to Lead/Support Districts:
Guided facilitation through the Assessment Inventory process;
A comprehensive communication plan to the Board of Education and
community;
Support in navigating the state and political climates of testing (macro-
level)
**The auditing of assessment practices should be ongoing.
52
What do you see as the pros and
cons of a state education agency
leading a student assessment
inventory process for school
districts?
Every Student Succeeds
Act
SEC. 1202. STATE OPTION TO CONDUCT ASSESSMENT SYSTEM AUDIT
Audit State assessment systems and ensure that local educational agencies
audit local assessments (not less than $1,500,000 per State)
purpose for which the assessment was designed and the purpose for
which the assessment is used;
legal authority for the administration of the assessment;
how teachers, principals, other school leaders, and administrators use
assessment data to improve and differentiate instruction;
the timing of release of assessment data;
the extent to which assessment data is presented in an accessible and
understandable format for all stakeholders;
ESSA
55
SEC. 1202. STATE OPTION TO CONDUCT ASSESSMENT SYSTEM AUDIT
opportunities, resources, and training to review assessment results and
make effective use of assessment data
distribution of technological resources and personnel necessary to
administer assessments;
the amount of time teachers spend on assessment preparation and
administration
ED has provided guidance for states and districts to use current funds (under
section 6111 for states and Title II for districts) to conduct assessment audits.
ESSA
56
Recommendations
Test Publishers
Be prepared to provide technical information to districts.
Make it easy for districts to get information and answers to questions about
assessments.
Consider partnering with districts to establish validity for intended uses that
do not have or have limited evidence, where appropriate.
Integrate your approach (i.e., collaborate).
Be aware of the state assessment picture.
Recommendations
58
States
Define your terms. They are key to having a conversation about
assessments.
Commit to transparency at all levels.
Create “space” for the inventory process.
School districts need support.
Signals importance and value of the work.
Integrate your approach (i.e., collaborate). Remember to use your partners
(e.g., RESAs).
Start with districts that are interested in conducting an inventory.
Recommendations
59
States
Provide a structure, but allow flexibility.
Support districts in conducting the process from where they are
starting.
Signal importance of process by taking close look at all state assessments.
Remember, uncertainty or flux with state assessments, presents
challenges for school districts in deciding to eliminate, modify, or keep
assessments.
From the beginning make this an embedded, ongoing activity rather than
an “once and done.”
Recommendations
60
Questions?
Comments
Your thoughts
61
For more information: www.achieve.org/assessmentinventory
◆ Neil Gupta, Director of Secondary Education, Worthington City Schools, Ohio |
◆ Kris Kaase, Independent Consultant | [email protected]
◆ Jacob Mishook, Associate Director, Assessment and Accountability,
Achieve | [email protected]
◆ Brian Touchette, Education Associate, Delaware Department of Education |
We are very interested in hearing your feedback.
Information and Contacts
62
