Exemplar Protocol for Most Formative Assessment - sccoe.org 2012... · Web viewGrade Level: 10th. Subject: Science. ... and tools for monitoring the improvement of student learning

Exemplar for Using Most Formative AssessmentsMay 24th, 2012

Team: Exemplary Team Grade Level: 10th Subject: Science

Introduction:Our team seeks to use Most Formative Assessments to support all students in achieving the Common Core State Standards. Most formative assessments are part of the Assessment Continuum. Teachers and students use Most Formative Assessments in the moment when instruction occurs and when a rapid collection of information, diagnosis, and intervention can be most helpful in supporting student learning.

Purpose:The main purpose of the Most Formative Assessment Module is to provide teachers with the knowledge and skills that they will need to implement most formative assessment practices that will support the collection of immediate information concerning student performance on lesson-based learning targets aligned to the Common Core State Standards. The module provides teachers with resources and tools that they can use to collect instantaneous information about all students performance on key learning targets; diagnosis; intervention to support groups or individual students; and tools for monitoring the improvement of student learning. This module will also provide teachers with technology tools that they can use to carry out most formative assessment routines within their classrooms. This module will provide both the moral purpose and research support for the effectiveness of most formative assessments in supporting student learning. Finally, the module will end with support for principals and district leaders in supporting the use of most formative assessment systems within classrooms and schools.

Module Outline:1. Introduction to the Assessment Continuum2. Definition of Most Formative Assessments3. Examples of Most Formative Assessments4. The Moral Purpose Supporting Most Formative Assessments5. Research Supporting the Use of Most Formative Assessments6. Most Formative Assessment Model7. Technology Support for the Use of Most Formative Assessments8. Leadership Support for the Use of Most Formative Assessments

Page 1

Exemplar for Analyzing Benchmark Assessments Santa Clara Unified School District

What problems are we trying to fix? How do we measure whether all of the students in our classrooms achieve the intended

learning targets while instruction occurs?

How do we use the assessment information to diagnose whole class, small group, and/or individual student learning needs and then intervene to support those learning needs?

How do we monitor how well our intervention supported student learning?

What is the Assessment Continuum? Most Formative Assessments

o Definition: Most formative assessments are closest to instruction. They take place within the actual instructional cycle and provide students and teachers with immediate feedback about the degree to which all students are achieving the instructional targets. Results from these assessments can be used to address student learning needs quickly.

o Examples: Minute to Minute Assessment Questioning by teachers and students Whiteboard responses to key questions

More Formative Assessments Definition: More formative assessments are used after a short instructional cycle

of 1-4 weeks. They provide students and teachers with feedback about student learning needs aligned to instructional targets. Results from these assessments can be used to address student learning needs in an ongoing way.

Examples:o Science Progress Assessmentso Teacher team developed common assessmento Performance Assessmentso NWEA Assessments

More Summative Assessments Definition: More summative assessments are used at the end of a longer

instructional cycle of 6-8 weeks and are meant to measure student achievement of learning targets after the student has had many opportunities to achieve and demonstrate mastery of the targets.

Examples: End of Unit Assessments Mid-Year Benchmark Math Placement Assessment

Most Summative Assessments

Page 2


Definition: Most summative assessments are usually used at the end of a semester or at the end of the year. Results can be used to make key decisions for students. Results are also used to inform the success of the adults in helping students achieve key learning standards.

Examples: California Standards Test (CST) Scholastic Aptitude Test (SAT) National Assessment of Educational Progress NAEP)

Activity 1: Based on your review of the Assessment Continuum, definitions, and examples, work individually and then in pairs to answer and discuss the following questions.

What are the purposes of the assessments within the Assessment Continuum? How are they similar? How are they different?

The purpose of formative assessments is to establish questions or tasks that will elicit student thinking about a learning target in order to diagnose possible student learning needs and address possible student misconceptions. Formative assessments also include an evaluative component in which teachers diagnose student learning needs and then implement interventions to address those learning needs. Teachers also monitor student achievement of the learning targets after the intervention is implemented.

Summative Assessments are designed to collect assessment information as well, but the information is used to inform educators about how well the system or key initiatives are operating to improve student learning. The evaluative aspects of summative assessments are to diagnose the needs of the system to improve student learning and much less on supporting individual students.

How are the audiences for the assessments within the Assessment Continuum similar and different?

The audiences for formative assessments are the students during the instructional process when learning is actually taking place. The audiences for summative assessment results are the educators, parents, and community who depend upon the educational system to improve student learning.

Can we expect one Assessment type to serve multiple purposes and audiences? Please discuss.

Assessments are rarely purely formative or summative. Assessments can sometimes serve both formative and summative purposes. For example, a school system might do an analysis of how students select distractors on a Benchmark Assessment in order to better diagnose student learning needs and design re-engagement lessons aligned to the measured learning

Page 3


needs of the students as evidenced by distractor selection on a Benchmark Assessment. The overall results of the Benchmark Assessment can also be used in a summative way to inform the success of an educational initiative aligned to the content assessed on the Benchmark Assessment. If the results are scaled in a similar way that the California Standards Test (CST) is scaled, the results can also be used to predict future performance on the California Standards Test (CST).

How do the Experts Define High Quality Assessments?Formative assessment occurs while knowledge is being learned. Summative assessment occurs at the end of a learning episode – for example, at the end of a course. (McMillan, 2000)

Formative assessments are interactive and used primarily to form or alter an ongoing process or activity. In contrast, assessments that come at the end of a process or activity, when it is difficult to alter or rectify what has already occurred, are called summative assessments. (Airasian, 1994)

Formative assessments are all those activities undertaken by teachers and/or by students which provide information to be used as feedback to modify the teaching and learning activities in which they engage. (Black and Wiliam, 1998)

Page 4


Comparing Formative and Summative Assessments

Activity 2: Based on your review of the Assessment for (Formative) and the Assessment of (Summative) graphic, please work with an elbow partner to discuss the following questions.

How are the foci of Formative and Summative Assessments similar? How are they different?

Some samples of different foci for Summative Assessments include: Curriculum Development Professional Development Supervision Program Evaluation Finances Equity Analyses Mid-Course changes to initiative implementation

Some samples of different foci for Formative Assessments include:

Page 5


Short-term Instructional diagnosis Mid-term Instructional diagnosis Coaching Differentiated Instruction Flexible student grouping Academic Conferences Teacher Collaboration Lesson Design

Summative assessments are different from formative assessments in that the information collected from summative assessments informs adults within the system as to how well the system is working in support of student learning.

Formative assessments directly support student learning while learning is taking place. The best use of formative assessments includes teacher reflection about student learning needs; diagnosis of student learning needs; interventions that specifically address student learning needs; and ongoing monitoring of student achievement of learning targets.

Which type of assessment contributes most directly to building student capacity to learn? What are some examples from your practice in using these kinds of assessments?

Because formative assessments occur with the instructional practice and are non-graded, they offer the most potential to support student learning. Some examples of formative assessments include:

In a lesson about building a geologic time scale in science the teachers developed the following learning targets:

Students can visually represent large numbers like a million Students can construct and use a ration to solve a problem Students can construct a proportion to solve a problem Students use their knowledge of ratio and proportion to build various kinds of geologic

time scales.After the end of each lesson, the teacher used the formative assessment called the Exit Ticket to gauge the degree to which her students achieved the learning targets. Annotations that accompany the Exit Ticket below demonstrate teacher diagnosis of student responses and possible next steps that she might take to address student learning needs.

Page 6

Page 7

Do summative assessments play a role in informing the quality of the system used to support student learning? Please describe how your system uses these types of assessment.

When used in conjunction with the implementation and careful monitoring of key instructional initiatives, summative assessments play an important role in informing the educators in a district or school about the whether the well implemented initiative is making a difference in supporting student achievement.

Instructional leaders within the Redwood City School District reviewed student performance on the California Standards ELA Test and found that about half of the second grade students were not reading at grade level. They used this information to plan and implement an early reading screening and diagnostic assessment system called the Dynamic Indicators of Basic Early Literacy or DIBELS. The visualization demonstrates the poor performance of 2nd grade students in ELA and the second page shows the detailed plan for screening and monitoring students in grades K- 3 for the critical early reading skills of phonemic awareness, phonics, fluency, vocabulary, and comprehension.

Page 8

Page 9


Page 10

A Model of Formative and Summative Assessments

Activity 3: Based on your review of the model for formative and summative assessments, discuss the questions below with a table partner.

Describe whether you support or do not support the idea of using at least 3 un-graded formative assessments for every summative assessment that is administered to students.

Teachers may argue for both positions. Examples of potential support for both positions are provided below.

Arguments that support not grading formative Assessments

Arguments that support grading formative assessments

When combined with focused formative feedback, non-graded assessments can support students in taking the next step toward improving their learning without worrying about the stigma of a grade.

In order to succeed, sometimes need to fail in order to learn how to improve their performance.

Occurs during instruction when student

Students need to be held accountable for the work that they do in class whether formative or summative. Grades help students become more responsible and accountable.

All assessments that a student participate in should help contribute to a grade.

Page 11


Arguments that support not grading formative Assessments

Arguments that support grading formative assessments

learning is happening. It is penalty free Research has shown that this kind of

assessment promotes profound achievement gains for all students with the largest gains for the lowest achievers.

Supports the developmental growth within a skills set or knowledge domain.

How do you think that un-graded formative assessments can contribute to improved gains in student learning?

Research by Hattie and others have shown that the use of formative assessment with descriptive feedback results in 0.6 effect size in student achievement. This would be like taking a child who is performing at the 50th percentile to the 80th percentile.

Ungraded formative assessment opportunities focus student attention on the intrinsic components of knowledge and tasks that they are trying to learn or perform and remove focus on extrinsic reward systems.

Can the summative assessments in the model provide any formative assessment value for students? Please describe.

Summative Assessments can sometimes provide formative information for teachers and students. Many times, benchmark assessments are built in such a way that the distractors provide formative information that teachers can use to inform student knowledge about error patterns in their thinking. The descriptions below were developed by the team at the Santa Clara County Office of Education.

By listening to students, we can identify error patterns in student thinking and then build assessment items that incorporate distractors that attract student responses where students may have these error patterns in thinking.

Page 12

Item 15 aligns with a component within Algebra I Standard 6.0, in which “students graph a linear equation and compute the x- and y- intercepts (e.g. 2x + 6y = 4). They are also able to sketch the region defined by linear inequality (e.g., they sketch the region defined by 2x + 6y < 4).”

Page 13


Page 14


Page 15

What are Formative Assessments?

Activity 4: Please use the Venn Diagram Below to identify similarities and differences between Formative (assessment for learning) and Summative Assessments (Assessments of Learning). Please consider purpose, audience, and use in your comparison.

Page 16

Summative Assessment

Formative Assessment

Used to diagnose student learning needs during instruction

Used to elicit student misconceptions

Used to screen and monitor student achievement

Supports student meta-cognition

Used to inform the system on how well it is supporting student learning.

Used for student promotion or graduation purposes

Interim Benchmark Assessments used to improve system and also conduct distractor analysis to inform instruction

What are some examples of Most Formative Assessments?

Activity 5: The Book by Page Keeley called Science Formative Assessment contains 75 high quality examples of most formative assessments. You will be given a packet of 3 examples of Most Formative Assessments. Please read each example and then think about how you might use the most formative assessment within the context of your own teaching. Please discuss the answers to the questions on the subsequent slide with a table partner.

Please describe a classroom instructional event where you might be able to use one of the Formative Assessment Classroom Techniques described in this book.

The lesson previously described would provide a great opportunity to conduct the most formative assessment called Agreement Circles found on page 51 of the Science Formative Assessments Resource by Page Keeley.

Students would form a circle and then provided with the statement. It will take more than one year for a bacterial colony to divide exponentially to reach the mass of the earth?

Another way to frame this question would be to focus on the division cycles that it would take for a bacterium to divide exponentially to become as massive as the earth.

Students who agree with the statement would stay on the outside of the circle and those who disagree would move to the center of the circle.

Students would then pair to discuss and defend their agreement or disagreement positions.

Key Questions:

• How would you embed the assessment into your instruction?

It would be good to use this most formative assessment in a pre- and post manner. Do students’ position and argumentation change after participating in the lesson? Most students will initially believe that it will take a long time with many cycles of divisions before a bacterium could divide and result in a colony as massive as the earth.

• What might be some responses that you might get from students?

Most students will report that it will take a long time for a bacterium to divide into a colony that would become as big as the earth. Used at the end of the lesson, this assessment would identify students who still maintained the misconception that exponential growth will take a long time.

• How would you use the information that you collected to diagnose student learning needs?

Page 17


A teacher could use this information to identify students who still maintained misconceptions about the characteristics of exponential growth and then provide new simulations that demonstrate the properties of exponential growth.

• What interventions would you use to address student learning needs?

Provide students with technology-based simulations that demonstrate exponential growth in a time-lapse manner.

• How would you monitor whether students learned?

I would continue to use Most Formative Assessments like the Agreement Circles to monitor that all students understand the properties of exponential growth.

What is the Moral Purpose for the Use of Formative Assessments

"Of all the civil rights for which the world has struggled and fought for 5,000 years, the right to learn is undoubtedly the most fundamental...The freedom to learn...had been bought by bitter sacrifice. And whatever we may think of the curtailment of other civil rights, we should fight to the last ditch to keep open the right to learn, the right to have examined in our schools not only what we believe, but what we do not believe; not only what our leaders say, but what the leaders of other groups and nations, and the leaders of other centuries have said. We must insist upon this to give our children the fairness of a start which will equip them with such an array of facts and such an attitude toward truth that they can have a real chance to judge what the world is and what its greater minds have thought it might be."

W.E.B. DuBois

Page 18


The True Meaning of Assessment

Activity 6: Please describe below how this painting exhibits the elements of a quality formative assessment experience.

The painting exhibits many of the positive attributes of high quality formative assessments including:

Opportunity for the teacher to effectively listen to and respond to student learning needs.

Ability of the teacher to provide just in time support for student learning. Coaching can be provided to the student during the act of playing resulting

in immediate improved performance. Support provided in a safe and nurturing environment. Support provided in a loving and caring manner

Page 19

The word assessment comes from the Latin root word assesare which means to sit beside. Your team will receive a copy of a painting by Tanner called the Banjo.


Instructional Practices that affect Student Achievement: What the Research Says

Source Characteristics of Feedback from Classroom Assessments

Number of Studies

Effect Size

Percentile Gain or Loss in Student

Achievement

Bangert-Drowns; Kulik, Kulik, & Morgan, 1991

Right or Wrong 6 -0.8 -3

Provide correct answer 39 0.22 8.5

Criteria understood by students vs not understood

30 0.41 16

Explain 9 0.53 20

Repeat until correct 4 0.53 20

Fuchs & Fuchs (1986)

Displaying results graphically

89 0.70 26

Evaluation (interpretation) by rule

49 0.91 32

Source: Marzano

Activity 7: There is abundant research that supports the use of Formative Assessment in supporting student learning. John Hattie conducted over 800 meta-Analyses on 138 on various influences on student learning. Providing formative evaluation to students was the third highest influence on student learning.

You will be provided with a reading from an important research student about the use of formative assessment. Please read these resources and discuss the questions on the following page with a table partner.

Research Sources:

Source 1: Visible Learning by John Hattie. Pages 173- 178

Source 2: Classroom Assessment and Grading by Robert J. Marzano. Pages 1-11.

Source 3: Inside the Black Box by Paul Black and Dylan Wiliam. Pages 1-6.

After reading your Research Source Material, please form groups of 3 and discuss the following questions with your team members.

Page 20


What are two or three key research findings that you found in the source?

Formative assessment is most effective when it is combined with effective diagnosis and just in time intervention (Black and Wiliam).

Providing feedback to students enhanced by a clear set of rules can result in a 32 percentile point gain in student achievement. (Marzano).

Feedback is most powerful when it moves from student to teacher as it makes learning visible to the teacher and the student. (Hattie)

Why are these findings important?

These findings are important because they can help define how formative assessment should be implemented to achieve the best outcomes for student achievement.

How can you use the findings in your practice of using formative assessments?

Teachers might identify potential student misconceptions and then develop assessments that will elicit those misconceptions within class.

Teachers might consider building in more opportunities for students to share their explanations for solutions to problems that involve important learning targets.

Teachers might establish clearer rules for the achievement of key learning targets a long with a rubric that students can use to self-assess their progress.

How can we model the use of Most Formative Assessments?

We can model the use of Most Formative Assessments by beginning with a Lesson Plan aligned to helping students demonstrate their understanding of learning targets that align with the Common Core Standards. We will begin this modeling process with a lesson entitled: Exponential Growth: How Long will it take for a bacterial colony to grow as big as the earth? Please review the copy of this lesson plan that is attached to this protocol.

Activity 8: Review the complete lesson plan and then discuss the following questions with a table partner:

Are the learning targets important for students to know? Why?

The ability to effectively use both ratio and proportion is a critical 21st Century skill required for both College and Career success. This lesson moves students in a cognitively appropriate way from the use of concrete manipulatives, to a conceptual understanding, to an application through a real world modeling experience using EXCEL.

Page 21


What are the opportunities for most formative assessments?

There are many opportunities to use formative assessments within this lesson. One example provided in the lesson is the use of Exit Tickets to gauge student understanding of the key learning targets in the lesson including the use of large numbers; scientific notation; ratio; and proportion.

How might you use most formative assessments to elicit student misconceptions?

The teacher could use the Agreement/Disagreement Formative Assessment tool to gauge whether students have misconceptions about the properties of exponential growth. Most students will think that it will take a very long time for one bacterium to divide into a colony that has the same mass as planet earth.

Activity 9: Because the size of a bacterium is so small, and the size of the earth is so large, students will probably believe that it will take a very long time for a single bacterium through repeated divisions to become as big as the earth. You can use the most formative assessment of the Crumpled Paper throw to elicit student understanding of exponential growth by asking them to record on a small piece of paper, how long that they think it will take for a bacterium through repeated divisions to become as big as the earth and tell also say why. You can find how to carry out this Most Formative Assessment strategy on the following information.

Commit and Toss Description

Commit and Toss is an anonymous technique used to get a quick read on the different ideas students have in the class. It provides a safe, fun, and engaging way for all students to make their ideas known to the teacher and the class without individual students being identified as having “wild” or incorrect ideas. Students are given a question. After completing the question, students crumple their paper up into a ball and, upon a signal from the teacher, toss the paper balls around the room until the teacher tells them to stop and pick up or hold on to one paper. Students take the paper they end up with and share the ideas and thinking that are described on their “caught” paper, not their own ideas.

Page 22

Page 23


Page 24

The weight of a bacteria is so small

and the weight of the earth is so great that it would take forever

for dividing bacterium to become

It will take millions of years for a

bacterium to become as big as the earth by


Page 25

Teacher Diagnosis: Students do not have a concept of the properties of exponential growth.

Intervention: Engage students in hands on and minds on activities that will help them visualize and conceptualize the properties of exponential growth. Students will use both physical models and Excel models of exponential growth.

Activity 9: Discuss with a table partner where it might be appropriate to do the Most Formative Commit and Toss assessment strategy.

The Commit and Toss activity would be great to ensure participation by all students and to guarantee anonymity of the students.

Activity 10: In order to model, exponential growth, students will need to be able to represent large numbers using scientific notation. Students may exhibit the following error patterns in representing large numbers using scientific notation.

What value below represents the number 0.00036

A. 3.6 x 104

B. -3.6 x 104

C. 3.6 x 10-4

D. -3.6 x 10-4

Page 26


Please produce an answer frequency table using SChoolPlan that shows a 42% selection of letter B and a 40% selection of letter C.

Page 27


Activity 11: The Exit Ticket is a Most Formative Assessment Tool to gauge how well students are achieving key learning targets.

How might you use this type of tool to measure student understanding of learning targets in your classroom?

How might you use this type of tool to measure student understanding of learning targets in your classroom?

This tool would be a helpful student metacognitive tool to help students gauge their own progress in achieving the two learning targets. This system could also cue the teacher as to which students still need support in achieving the learning target(s).

The cards could be sorted for differential grouping on the following day to re-teach modeling bacterial growth and/or using scientific notation.

Page 28


How does Leadership Support for the Use of Most Formative Assessments?

The Fixsen Model for Implementation supports a system that effectively combines Leadership, organizational supports, and staff competencies to ensure an effective implementation of a Most Formative Assessment Initiative.

Page 29

http://nirn.fpg.unc.edu/resources/implementation-research-


http://nirn.fpg.unc.edu/resources/implementation-research-synthesis-literature

School and District leaders work collaboratively to build the capacity of teachers to implement quality initiatives like the use of Most Formative Assessments within the Classroom. Leaders do this by focusing not only on the technical capacities of staff to effectively implement the initiative but also those transformational skills required to solve problems that arise that are not easily characterized or defined.

Activity 12: Please read the article called Implementation Drivers - Best Practices for Coaching by Karen Blasé et al. Please review the checklist for the implementation of quality coaching for the initiative and discuss the following questions with your table partner.

Does the document provide enough information to implement a Coaching system for the implementation of an innovation?

The Implementation Drivers can help the Implementation Team think about the coaching model and how they can help develop high quality effective, coaching services and supports that

Page 30


contribute to program and practice effectiveness. They can also “score” each Driver in terms of how focused the Driver is on Implementation.

The Team using the Checklist also will want to discuss the importance and perceived cost-benefit of fully utilizing the best practices related to each Driver as well as the degree to which the Team has „control‟ over each Driver and the associated „best practices‟. When the best practices cannot be adhered to, then the Team needs to be confident that weaknesses in one Driver are being compensated for by robust application of other Drivers to produce and sustain high quality, effective coaching. For example, if skill-based training is not offered with qualified behavior rehearsal leaders who know both the intervention well and the coaching processes, then there will need to be a greater emphasis on the Selection of Coaches and on providing support and feedback for coaches in the field, that is coaching for coaches, in order to “compensate” for the weakness in the Training Driver (e.g. lack of practice with skilled feedback providers).

As a district leader, how would you modify this tool to monitor the implementation of the Most Formative Assessment Initiative?

The tool provides a clear recognition of all of the implementation elements that need to be put into place to ensure a successful implementation. The level of detail is sufficient to ensure a quality implementation of only one aspect of the implementation model – Coaching.

Page 31


References

Black, Paul and Wiliam, Dylan. 2001. Inside the Black Box. Raising Standards through Classroom Assessment. King’s College London School of Education.

Blasé, Karen. Et al. 2009. Implementation Drivers – Best Practices for Coaching. National Implementation Research Center.

Fixsen, Dean. Et al. 2005. Implementation Research. A Synthesis of the Literature. University of South Florida.

Hattie, John. 2009. Visible Learning. A Synthesis of Over 800 Meta-Analyses Relating to Achievement. Routledge. New York. New York

Keeley, Page. 2008. Science Formative Assessments. Corwin Press. Thousand Oaks, California.

Marzano, Robert, J. 2006. Classroom Assessment and Grading that Work. ASCD. Alexandria, Virginia.

Mason, Dan. Mark Moulton, Ph.D., Dale Russell, Ed.D., Diana Wilmot, Ph.D., 2009. Three Facets of Formative Assessment. Santa Clara County Office of Education. San Jose, CA.

Page 32

Documents

Exemplar Protocol for Most Formative Assessment - sccoe.org 2012... · Web viewGrade Level: 10th. Subject: Science. ... and tools for monitoring the improvement of student learning