Checking the ABC’S – Systematic Error Analysis in Student Assessments Scott Clark Molly Barrett...

Checking the ABC’S – Systematic Error Analysis in

Student Assessments

Scott Clark

Molly Barrett

CAIS 2007

Agenda

• History

• How we use this system

• Helpful hints for implementation

• Justification

• Practice

• What are others doing with this type of work?

History…Evolution of Error Coding

• Nov. 2005 article by Grant Wiggins and Stan Izen in Independent Teacher magazine on testing and assessment:

– “…we need to make sure that students get feedback that they can use and profit from, [and] opportunities to use that feedback…”

• Four main categories: ABC’S

The Grid!

A - Assessment Day Issue

• Running out of time on test

• Misreading the directions or problem

• Making careless “OOPS!” errors

• Blanking out on a problem

• Not showing work! Not communicating ideas clearly

• Leaving problems blank by mistake

B = Background Topics/Skills Issue

• Forget math from previous years

• Forget math from previous chapters

• Weak in algebra skills

C = Current Topic/Skills Issue

• Missing basic facts - losing points on true/false, matching, or fill-in-blank

• Missing routine problems - losing points on problems that were just like homework and classwork only with different numbers

• Missing calculator skills - losing points for not knowing how to use the calculator (usually applies to graphing calculator skills)

S = Synthesis

You are losing points on problems that present new challenges or situations. You are struggling to bring to bear your knowledge and talent on a problem that is slightly or even significantly different from those you saw in class.

How We Use This System

• Purpose explained to students in Sept.

• When grading test, teacher writes letter of type of mistake next to problem, and might circle where the mistake was made.

• Using coded test, the student completes test corrections. Part of the correction process - identify the specific type of error (e.g. A2).

• When test corrections are turned in, 2 minutes are spent filling in the error coding grid. Teacher keeps grids until the next quiz/test.

Test Correction Guidelines Complete all corrections on separate paper.

Mark error coding on each corrected problem (ex. A2, B1, etc..)

Correct each problem that had a mistake. Show all work. Except for word problems, recopy all problems that are to be revised.

For word problems, recopy the question and the essential information (you do not need to copy the entire problem).

Revise the solutions so that they are correct.

Place a circle around revised answers.

At the end of each problem, describe in clear, written detail why the mistake occurred, and what has been learned in the revision process.

Sample Test Corrections:

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Sample Test Corrections:

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

This problem, I made a C2 mistake. I made a mistake on the original test because, instead of subtracting first, I divided. In the future, I will remember to use SADMEP on my problems.

Implementation Hints

• Copy grid and key on colored paper to make them easier to find later.

• Teacher should keep the grids so a consistent record can be kept.

• 3-4 times a year (or before midterm/final) have students make observations about any patterns in their errors.

• Refer students back to the advice part of the key – how to improve!

Justification

THINKING SKILLSCONTENT

PROCESSES

HABITS OF MINDFOUR LEVELS OF OUTCOMES:

http://www.habits-of-mind.net/ppt/1BuildaThoughtfulLrnCommHOM.ppt.Costa & Kallick (2000). Discovering and Exploring Habits of Mind, Alexandria, VA: ASCD, p55.

ACTIVITIES

Habits of Mind• Persisting• Managing impulsivity• Listening with understanding

and empathy• Thinking flexibly

• Thinking about thinking (metacognition)

• Striving for accuracy• Questioning & posing

problems• Applying past knowledge to

new situations

• Thinking & communicating with clarity and precision

• Gathering data through all senses

• Creating, imagining & innovating

• Responding with wonderment & awe

• Taking responsible risks• Finding humor• Thinking interdependently• Remaining open to

continuous change

Habits of Mind• PersistingPersisting• Managing impulsivityManaging impulsivity• Listening with understanding

and empathy• Thinking flexibly

• Thinking about thinking Thinking about thinking (metacognition)(metacognition)

• Striving for accuracy• Questioning & posing

problems• Applying past knowledge to

new situations

• Thinking & communicating with clarity and precision

• Gathering data through all senses

• Creating, imagining & innovating

• Responding with wonderment & awe

• Taking responsible risks• Finding humor• Thinking interdependently• Remaining open to

continuous change

Metacognition

• Thinking about thinking– Is this an A error or a C error? Students make

the final call (but often need teacher oversight)

– What kind of A error is this?– Are there patterns that I observe in the kinds

of errors that I am making?– What can I do differently to improve my

performance?

Student Perspectives

• Error coding helps me to see what I have done wrong on previous tests. This helps because I make a checklist in my mind of what I need to do, to make sure that I don’t make them again. - Kyle

• Error Coding: very useful because it tells me what mistakes I make most often. I had no idea that I made so many careless mistakes. Now I check my work more accurately. - Amara

Student Perspectives

• Error coding helps me realize what kinds of mistakes I make on test, and how I should fix them, instead of just getting points off with no explanation of why. - Victoria

•Error coding gives students a chance to see their accumulated mistakes without having to keep track of each test. - Kate

The Head-Royce Experiment in Grading Consistency

(our own practice of metacognition)

Practice

“Fake error coding” today:

– Look at the problem and the sample response

– Using the error coding key, write the letter of the type of mistake you think the student made

– Talking is encouraged!

What are other schools doing with this type of work?• Habits of Mind in Curriculum Maps (JRPO schools)• Common scoring key (ex. HRS Algebra midterm)• Checklists for self-monitoring (across curriculum, work

habits) - check w/school learning specialists• Apply similar system to homework• Model hwk answers - what you want it to look like• Oops errors - taking off 0 points• Don’t tell students whether or not answer is correct -

student must convince class of solution.• Color coding grading - (green - correct; blue - can

corrected for full credit; red - can be corrected for half credit)

• Bring note card to next test based on previous error coding.

• Include test-anxiety related section on grid

Evaluation

• Please give us feedback!

• Feel free to contact us:– Scott Clark: sclark@headroyce.org– Molly Barrett: mbarrett@headroyce.org