Pennsylvania Training and Technical Assistance Network Embedding Formative Assessment : Five Key Strategies and One Big Idea February 10, 2012 Facilitator:

Pennsylvania Training and Technical Assistance Network

Embedding Formative Assessment : Five Key Strategies and One Big Idea

February 10, 2012

Facilitator:DonnaIrene McKinley

PaTTAN’s Mission

The mission of the Pennsylvania Training and Technical Assistance

Network (PaTTAN) is to support the efforts and initiatives of the

Bureau of Special Education, and to build the capacity of local

educational agencies to serve students who receive special

education services.

PDE’s Commitment to Least Restrictive Environment (LRE)

Our goal for each child is to ensure Individualized Education Program (IEP) teams begin with the general education setting with the use of Supplementary

Aids and Services before considering a more restrictive environment.

Anticipation Guide

• Find your Anticipation Guide in your Supportive Handouts

• Complete on your own.• After reading each classroom

scenario place a plus (+) next to a scenario that demonstrates formative assessment instructional practices

• or a minus (-) next to a scenario that does not.

Handout

PA’s Standards Aligned System

Standards

Assessments

CurriculumFramework

Instruction

Materials & Resources

Safe and Supportive

Schools

StudentAchievement

8

Assessments aligned to PA standards.

Summative

FormativeBenchmark

Diagnostic

Standards

Assessments

Instruction

Materials & Resources

Safe and Supportive

SchoolsStudent

Achievement

Curriculum Framework

Standards Aligned System (SAS) Assessments (http://www.pdesas.org/)

9

Dylan Wiliam, PhD

Why Raising Achievement Matters!

Why Raising Achievement Matters

12

Handout

Refer to your Video-Note Taking Guide for a Prompt

Foundational Concepts

NOW more than ever…

$0.00

$5.00

$10.00

$15.00

$20.00

$25.00

$30.00

$35.00

1973

1975

1977

1979

1981

1983

1985

1987

1989

1991

1993

1995

1997

1999

2001

2003

2005

Dropout

HS Diploma

Some College

BA/BSc

Prof Degree

[Source: Economic Policy Institute] 13


Which of the following skill categories is disappearing from the work-force

most rapidly?

Fist-to-Five

1. Routine Manual2. Non-Routine Manual3. Routine Cognitive4. Expert Thinking & Problem-Solving5. Complex Communication

14


Accountable Talk (Resnick, 2000)

Agreements for partner conversations:•1. Stay on topic.•2. Use information that is accurate and appropriate to the topic.•3. Think deeply about what the partner has to say. (Fisher & Frey, p. 23. 2007).


Indicators of Accountable Talk1. Clarification

“Could you describe what that means?”

2. Justification “Where did you find that information?”

3.Challenge misconceptions “I don’t agree because…”

4. Demand evidence “Can you give me an example?”

5. Interpret and use each other’s statements “David suggested….”

(Fisher & Frey, p. 23, 2007)


The most salient information for me so far in this training is… because…..”

Where’s the solution?

17


It’s the Classroom

Variability at the classroom level is up to 4 times greater than at school levelPersonal Reflection:–What distinguishes the most effective from the least effective classrooms?–What differences would you see if you were in a classroom that took 6 months to learn versus a year?

18


What do we mean by Assessment for learning?

What Do We Mean byAssessment for Learning (AfL)?


20

Handout


Assessment for Learning

Assessment becomes formative when the evidence is actually USED to ADAPT the teaching in order to meet student learning needs.

Effect Size

“The mean effect sizes for most of the studies were between .40 and .70; such effect sizes are among the largest ever reported for sustained educational interventions.”

Assessment for Learning, 2003


The Good News

• An effect size of 0.40 would mean that the average pupil involved in an innovation would record the same achievement as a pupil in the top 35% of those not involved.”

• An effect size of 0.70 in the recent international comparative studies in mathematics would have raised the score of a nation in the middle of the pack of 41 countries to one of the tope five.”

Inside the Black Box, 1998


Black and Wiliam Meta-Analysis

• Over 160 research journals and books• Yielded about 681 articles or chapters• Research review based on 250 of these

sources“There is a body of firm evidence that formative

assessment is an essential component of classroom work and that its development an raise standards of achievement. We know of no other way of raising standards for which such a strong prima facie case can be made.” (Inside the Black Box, 1998).


EFFECTS of Formative Assessment

Long-cycle

– Student monitoring– Curriculum alignment

Medium-cycle

– Improved, student-involved, assessment

– Improved teacher cognition about learning

Short-cycle

– Improved classroom practice– Improved student engagement 25


Where the learner is going

Where the learner is How to get there

Teacher

Clarify and share learning intentions

Engineering effective

discussions, tasks and activities that elicit evidence of

learning

Providing feedback that

moves learners forward

PeerUnderstand and share learning

intentions

Activating students as learningresources for one another

Learner

Understand learning intentions

Activating students as ownersof their own learning

Aspects of Formative Assessment


Evidence/rationale for investment in formative assessments

“It is impossible to disentangle the impact of formative assessment from the instruction that follows it. When improved student outcomes are used as the criterion for effective practices, evaluations of classroom-based formative assessment are as much an evaluation of the instructional adjustments resulting from the assessments as they are of the assessments themselves.”

(Torgesen & Miller, 2009)


Quick Write

Take Ten Steps

Pair- Discuss:

Quick Write: The difference between assessment of learning and assessment for learning.

Then take your summary and walk ten steps to someone you do not know and discuss your summaries.


Classroom Assessment

Defining Formative Assessment

• Read: Classroom Assessment: Minute by Minute, Day by Day

By Siobhan Leahy, Christine Lyon, Marnie Thompson and Dylan Wiliam

Educational Leadership, November 2005, Volume 63, Number 3 Assessment to Promote Learning

Pages 19-24 30


Jigsaw Directions• Number off at your table 1-5

Everyone reads At the beginning of the article read:

• Introduction• Changing Gears• Our Work with Teacher

At the conclusion of the article read:• Using Evidence of Learning to Adapt

Instruction• Supporting Teacher Change 31

Handout


Jigsaw The Strategies

• 1’s read “Clarify and Share Intentions & Criteria”• 2’s read “Engineering Effective Classroom Discussions”• 3’s read “Provide Feedback that Moves Learners

Forward”• 4’s read “Activate Students as Owners of their

Learning”• 5’s read “Activate Students as Instructional Resources

for One Another”Share summary of your section and then have a

table discussion of how embedding formative assessment strategies into classroom practice may impact your teaching and your students’

learning


Text Rendering- On The Clock

What resonates with you ?

One sentence…

One phrase…

One word…


Going Deeper

A Focus on 5 Key Learning Strategies

and One Big Idea

34

5 Key Learnings

1. Clarifying, Sharing, and Understanding Learning Intentions

2. Engineering Effective Classroom Discussions, Activities, and Learning Tasks

3. Providing Feedback that Moves Learning Forward

4. Peer Assessment5. Self Assessment

5 Key Learning Strategies

35

5 Key Learnings

One Big Idea

Use evidence about learning to adapt teaching and

learningto meet student needs.

5 Key Learnings

1. Clarifying, Learning Intentions and Success Criteria

Common Language

Learning ObjectivesLearning

OutcomesLearning Goals

Learning Intentions

5 Key Learnings

Introduction: 1. Clarifying, Sharing, and Understanding Learning Intentions and Success Criteria

• Watch Wiliam Clip


40

Handout

5 Key Learnings

Key Point1. Clarifying, Sharing, and Understanding Learning Intentions and Success Criteria

Not all students have the same idea as their teachers about what they are meant to be doing in the classroom. Often, what is wanted is not made clear and this puts some students at considerable advantage because they already know. (Wiliam, 2011. p. 52).

41

5 Key Learnings

Examples:1. Clarifying, Sharing, and Understanding Learning

Intentions

5 Key Learnings

• Student-friendly language while maintaining the language of the

discipline

• Transferability- learning intention can be used in many other situations

A Good Learning Intention

A good learning intention is: 1.Important

2.Student-Friendly3. Clear

43

5 Key Learnings

Wallpaper Objectives- A Non-Example

1. Clarifying, Sharing, and Understanding Learning Intentions

44

5 Key Learnings

“Wallpaper” Objectives •Opaque to students

•Do not transfer to novel situations

I Do We Do You Do1. Clarifying, Sharing, and Understanding Learning

Intentions

Focusing on TransferabilityA Good Learning Intention is Clear

And focuses on what students will learn

NOT DO

45

5 Key Learnings

Non-Examples: Sharing Learning Intentions The Ambiguous Role of Context

Confused Learning Objective

Clarified Learning Objective

Context of Learning

To be able to write instructions on how to change a bicycle tire

To be able to present an argument for or against assisted suicide

To know what the local rabbi does

To produce and analyze a questionnaire about movie-going habits

To design an experiment to find out what conditions pill bugs prefer

46

Handout

(Wiliam, p. 61, 2011)

Random Reporter

5 Key Learnings

To be able to write clear instructions.

Changing a bicycle tire.

To be able to present and argument for or against an emotionally charged proposition.

Assisted suicide.

Graffiti Wall & Gallery Walk

• Think about a learning intention that you teach; write it on post-it. Make it context free.

• Share it with someone.• Place your post note on one of the 3

frames.

1.Elementary2.Middle School3.High School

5 Key Learnings

1. Clarifying, Sharing, and Understanding

Success Criteria

Examples:•A continuum of work samples

•Co-constructing with students the success criteria

What it looks like in the classroom•Student’s analyzing a continuum of

work samples•Co-constructing with students the

success criteria (rubric, performance levels)

5 Key Learnings

Time to Reflect…..

• In your current assignments, discuss to what extent you are seeing “visible objectives?”

• Are they posted?

• Are they being taught?

• Are they connected to the standards

• Are they connected to instruction/learning?

• Are students able to share what they are learning and why?

5 Key Learnings

Find the Fib: Pinch Card

A. Learning intentions should be important, clear, and student-friendly.

B. Learning intentions are synonymous with PA anchors and standards.

C. Students do better when they construct their own test questions and answers.

50

5 Key Learnings

Instructional Routine Card & Action Plan

Review Key Strategy 1. Clarifying, Sharing Learning Intentions and Success Criteria Instructional Routine Card

Write on your Action Plan.What one strategy is worth repeating?

Handout

5 Key Learnings

2. Effective Classroom Discussions, Activities & Learning Tasks

2. Engineering Effective Classroom Discussions, Activities and Learning Tasks


Refer to your Video-Note Taking Guide

for a Prompt

53

5 Key Learnings

Review Effective Discussions2. Engineering Effective Classroom Discussions, Activities, and

Learning Tasks

54

5 Key Learnings

Key Idea2. Engineering Effective Classroom Discussions, Activities, and

Learning Tasks

“…with careful planning and the thoughtful application of [questioning] techniques, the teacher can make the classroom a much more engaging place for students and one in which the teacher is able to make rapid and effective adjustments to meet the learning needs of all students.” (Wiliam, 2011, p. 104).

5 Key Learnings

2. Engineering Effective Classroom Discussions, Activities, and Learning Tasks

Only two good reasons to ask questions in class: 1.to cause thinking and 2.to provide information for the teacher about what to do next.

(Wiliam, p. 79, 2011)

56

5 Key Learnings

Bloom’s Taxonomy of Critical Thinking

New: Verbs Old: Nouns

Handout

Use questioning techniques to stimulate student thinking and provide information for the teacher about what to do next. (Wiliam, 2011).

Examples•Devote planning time and professional development to constructing questions that will engage and assess learning.•Use questioning to help students develop their ability to use language as a tool for thinking about content, both individually and in collaboration with other students. •Use open-ended questions that encourage students to discuss learning from multiple perspectives.•Listen to students’ responses interpretively (what you learn about students’ thinking by attending carefully to what they say)•Focus on the quality of talk, not the quantity of talk.

58

5 Key Learnings

Avoid frequent questioning techniques that are simply managerial questions or basic recall questions. (Wiliam, 2011).

Non-Examples•“Who has finished all the questions?”•“How many legs does an insect have?”•Listening evaluatively (listening to students’ answers to learn only what you want them to know).

59

5 Key Learnings

Practical Techniques for Questioning

60

• Causing thinking and generating data to inform teaching

–generating questions with colleagues

–closed vs open

–low-order vs high-order

–appropriate wait-time

5 Key Learnings

Practical Techniques for Questioning

61

• Eliminating I-R-E (initiate, respond, evaluate) and replacing with

–basketball rather than serial table-tennis

–‘No hands up’

–class polls to review current attitudes towards an issue

–All student response systems

5 Key Learnings

Socratic Questioning and Dialogue

•Ask questions that: clarify, probe purpose, probe assumptions, probe information, reasons, evidence and causes, probe concepts, inferences and interpretations.

•Ask questions about: viewpoints or perspectives, implications and consequences, questions about the question. (Foundation for Critical Thinking, www.criticalthinking.org)

5 Key Learnings

Socratic Seminar

• Conducting and engaging students in collaborative, intellectual dialogue facilitated with open-ended questions about a text to improve the individual’s ability to explain and manipulate complex systems.

5 Key Learnings

Ways to Overcome “I Don’t Know”

• Who Wants to Be a Millionaire Techniques– Phone a Friend– Class Vote– 50/50

• “If you did know, what would you say?”

• “If you were a mathematician (scientist/historian, etc.), what would you say?”

• Teacher feeds an answer; student repeats.

5 Key Learnings

Ways to Overcome “I Don’t Know”

• Pass Card– “Think about it, and I’ll be back.”– “Check in the text, and I’ll be back.”– “Check with a partner, and I’ll be

back.”– T: “ I’m back.” S: “ I still don’t know.”

T: “Which answer that you heard do you like best? Why?”

5 Key Learnings

Socratic Seminar

• Read the Gettysburg Address

Lincoln opens the address by saying that America is a nation “conceived in liberty and dedicated to the proposition that all men are created equal.” He follows by describing the Civil War as “testing whether … any nation so conceived and so dedicated can long endure.” What does he mean? What forces would prevent a nation dedicated to equality from surviving?

Why does Lincoln say that they “cannot dedicate, … cannot consecrate, … cannot hallow this ground” when that is precisely the purpose of the ceremony?

5 Key Learnings


Review Key Strategy 2. Engineering Effective Classroom Discussions, Activities, and Learning Tasks Instructional Routine Card

Write on your Action Plan. What one strategy is worth repeating?

Handout

5 Key Learnings

3. Feedback that moves learning forward

3. Providing Feedback that Moves Learning Forward


Refer to your Video-Note

Taking Guide for a Prompt

70

Handout

5 Key Learnings

Key points on Feedback3. Providing Feedback that Moves Learning Forward

• Feedback should always cause more work for the students (learners) than it does for the one who gives it

• Feedback should cause thinking• A “recipe” for future action• The rearview mirror versus the

windshield

71

5 Key Learnings

ExamplesExamples3. Providing Feedback that Moves Learning

Forward

1. Feedback that is focused on the learning not the student.

2. Feedback that is clear about what the student has achieved and what still needs further work to improve.

3. Feedback that points to how. 4. Feedback should be phrased as

targets.

5 Key Learnings

© PMB 2007

• the desired goal

• some understanding of how to close the gap

Feedback should provide:

Characteristics of Formative Feedback3. Providing Feedback that Moves Learning Forward

5 Key Learnings

• evidence on where they are now

Formative Assessment requires

• Data on the – actual level of some measurable

attribute– reference level of that attribute

• A mechanism – for comparing the two levels – information can be used to alter the

gap.

5 Key LearningsIf I had

known they wanted me to use all

this info., I would

never have asked for it!

Non- Examples3. Providing Feedback that Moves Learning Forward

• Frequent feedback is not necessarily formative

• Feedback that causes improvement is not necessarily formative

Assessment is formative only if the information fed back to the learner is used by the learner in making improvements

To be formative, assessment must include a recipe for future action

5 Key Learnings

Feedback Activity: Is it Formative?

5 Key Learnings


Review Key Strategy 3. Providing Feedback that Moves Learning Forward Instructional Routine Card

Write on your Action Plan. What one strategy is

worth repeating?

Handout

5 Key Learnings

4. Self Assessment5. Peer Assessment

4. Self and 5. Peer Assessment



79

5 Key Learnings

Key points 4.Self and 5.Peer Assessment

• Teachers can design learning situations, but only learners create learning

• If students can improve how they manage learning, they become owners of their own learning

• Back to the concepts of engagement and contingency.

80

5 Key Learnings

4.Self and 5.Peer Assessment

Proactive Metacognitive Learning

•Activating students as owners of their own learning

Collaborative Learning•Activating students as instructional resources for one another.

5 Key Learnings

4.Self Assessment

“Students can achieve a learning goal only if they understand that goal and can assess what they need to do to reach it. So self-assessment is essential to learning.”

Working Inside the Black Box, 2004

5 Key Learnings

Traffic Lights5 Key Learnings

Students reflect and indicate:

Red= Little understanding

Yellow= Partial understanding

Green = I understand

Another Example: :Learning Logs

Today I learned…I was surprised by….

The most useful thing I will take from this lesson is…

I was interested in…One thing I’m not sure about….

The main thing I want to find out more about…After this session, I feel…

I might have gotten more from this lesson if…

(Wiliam, pp. 157-158, 2011)

5 Key Learnings

5. Peer Assessment

Effective cooperative learning requires the presence of two elements:1.Group goals “Working as a group, not merely in a group.”

2. Individual accountability“Individual accountability, so that individual students cannot be carried along by the work of others.”

(Wilam, p. 135, 2011)

5 Key Learnings

Pre-Flight Checklists

1. How do these pre-flight checklist support interdependency?

2. How does having a clear learning intention and clear success criteria support peer assessment? How about feedback and questioning?

5 Key Learnings

4. Self and 5. Peer Assessment Techniques

• C3B4ME• Peer Homework

evaluation• Homework Help Board• Two Stars and a Wish• End of Topic

Questions• Error Classification• End of Lesson Student

Review

• What Did We Learn Today?

• Student Reporter• Preflight Checklist• I—You—We Checklist• Reporter at Random • If You Learned It,

Help Someone Who Hasn’t

5 Key Learnings

Practical Techniques:

• Conferencing and dialogue with students.

• Provide consistent opportunities for self-assessment.

• Learning Portfolios• Learning Logs• Traffic Lights• Colored Cups

88

5 Key Learnings

4.Self and 5. Peer Assessment

Tip the scales in the right direction:1.Share learning goals.2.Expect high expectations and help students believe they can achieve those goals. 3.Make it difficult for students to compare themselves with others.4.Provide a recipe for future action rather than a review of past failures. 5.Use every opportunity to transfer executive control from teacher to students.

5 Key Learnings


Review Key Strategy 4. Self and 5. Peer Assessment Instructional Routine Card

Write on your Action Plan. What one strategy is

worth repeating?

Handout

5 Key Learnings

5 Key Strategies: Are there Bigger Connections?

Sharing Learning Intentions• Curriculum

Questioning• Interactive Whole-Group Instruction

Feedback • Moving Student Learning Forward

Self and Peer Assessment• Attribution & Motivation• Reciprocal Teaching & Peer Tutoring

91

5 Key Learnings

Foundations of Professional Learning Communities

Teacher Learning CommunitiesProfessional Learning Communities TLCs/PLCs

• “The whole idea of TLCs is to give you (the teacher) a forum for getting support for putting in place your plans.”

Dylan Wiliam

Theory to PracticeTheory to Practice

That’s what TLC/PLC’s are for:

• Contradicting teacher isolation• Re-professionalizing • De-privatizing• Offering a steady source of support • Growing expertise • Facilitating sharing • Building a collective knowledge base

94


Pulling it all together

Anticipation Guide Revisited

• Take a moment to revisit the Anticipation Guide with which we began the day.

• Have any of your thoughts changed?

• If so, turn to your partner and tell why you would change your mark for a specific statement(s).


Frayer GO: Summarize TodayTheory to PracticeTheory to Practice

Frayer GO: Summarize TodayTheory to PracticeTheory to Practice

Putting it into Practice

Action Plan

Review your notes on your action plan.

Choose one!

Who will be your accountability partner?


Formative Assessment and Engagement Strategies Used in

this Training

1. Anticipation Guide2. Graphic Organizer3. Video Note Taking Guide4. Action Plan5. Fist to Five6. Accountable Talk7. Quick Writes


Formative Assessment and Engagement Strategies Used in

this Training8. Take Ten Steps9. Jigsaw10.Text Rendering11.I Do, We Do, You Do12.Random Reporter13.Graffiti Wall14.Gallery Walk


Formative Assessment Strategies

Used in this Training15. Reflection Time 16. Pinch Card17. Socratic Seminar18. Traffic Light19. Every student response plates/page protectors20. Frayer Graphic


Thank you!“In a completely rational society, the best of us would aspire to be teachers and the rest of us would have to settle for less, because passing civilization along from one generation to the next ought to be the highest honor and highest responsibility anyone could have.”

—Lee Iacocca

References

1. Black, P. & Wiliam, D. (1998). Inside the Black Box: Raising Standards Through Classroom Assessment. Phi Delta Kappan, October, 139-148.

2. Black, P., Harrison, C., Lee, C., Marshall, B. & Wiliam, D. (2004). Working Inside the Black Box: Assessment for Learning in the Classroom. Phi Delta Kappan, 86 9-2

3. Leahy, S. & Wiliam, D. (2009). Embedding Formative Assessment: A Professional Development Pack for Schools. Specialist Schools and Academies Trust.

4. Wiliam, D. (2011). Embedded Formative Assessment. Bloomington, IN : Solution Tree Press.

Contact Information www.pattan.net

DonnaIrene McKinleyEducational [email protected]/826-6859

Commonwealth of PennsylvaniaTom Corbett, Governor

Pennsylvania Department of EducationRonald J. Tomalis, Secretary

Dr. Carolyn Dumaresq, Deputy SecretaryOffice for Elementary and Secondary

Education

John J. Tommasini, DirectorBureau of Special Education

Patricia Hozella, Assistant DirectorBureau of Special Education

Documents

Pennsylvania Training and Technical Assistance Network Embedding Formative Assessment : Five Key Strategies and One Big Idea February 10, 2012 Facilitator: