Reviewing UBD and Moving Deeper into Assessment€¦ · ©2012, Karen L. Erickson. Do not duplicate without permission. Reviewing UBD and Moving Deeper into Assessment with Karen

©2012, Karen L. Erickson. Do not duplicate without permission.

Reviewing UBD and

Moving Deeper into Assessment

with Karen L. Erickson

Art Institute of Chicago September 15, 2012

1227C Central Street, Evanston, IL 60201 Phone: 847-328-6393, Fax: 847-328-6459

Website: www.creativedirections.org Email: [email protected]

http://www.creativedirections.org/

mailto:[email protected]


Fuzzy Foggy Fifty-Five

Students will become: able to acquainted with adjusted to capable of

cognizant of comfortable with conscious of familiar with

interested in knowledgeable about mature self-confident

Students will: admire come to know comprehend conceptualize create a classroom

atmosphere discover experience explore examine

hear know learn listen mind motivate perceive realize recognize

reduce immaturity review satisfy drives – needs see self-actualize study think understand value

Students will evidence a/an: appreciation for attitude of awareness of

comprehension of enjoyment of feeling for

interest in knowledge of understanding of

Students will exhibit: capacity depth

emotional maturity intelligence

purpose


The Functional, Forceful, Four Hundred Forty-Five Verbs to Use

Creative Behaviors alter generalize question regroup reorder restructure simplify ask modify rearrange rename rephrase retell synthesize change paraphrase recombine reorganize restate rewrite systematic design predict reconstruct

Complex, Logical, Judgmental Behaviors analyze compare criticize defend formulate induce plan appraise conclude decide evaluate generate infer structure combine contrast deduce explain

General Discriminative Behaviors choose describe discriminate indicate match order point collect detect distinguish isolate omit place select define differentiate identify list

Social Behaviors accept argue dance forgive invite participate smile agree communicate disagree greet join permit talk aid compliment discuss help laugh praise thank allow contribute excuse interact meet react volunteer answer cooperate

Language Behaviors abbreviate call indent punctuate sign summarize verbalize accent capitalize outline read speak syllabicate whisper alphabetize edit print recite spell tell write articulate hyphenate pronounce say state translate

Study Behaviors arrange circle diagram itemize map organize sort categorize classify find label mark quote underline chart compile follow locate name record cite copy gather look note search

Physical Behaviors arch chase grasp kick pull skip swim bat climb grip knock push somersault swing bend face hit lift run stand throw carry float hop march skate step toss catch grab jump pitch ski stretch walk

Arts Behaviors assemble construct fold illustrate paste sand stick blend cut form melt pat saw stir brush dab frame mix pour sculpt trace build dot hammer mold press shake trim carve draw handle nail roll sketch varnish color drill heat paint rub stamp wipe


Drama Behaviors act cross emit leave pass present sit build design enter memorize perform proceed start clasp direct exit move plan respond strike create display express pantomime practice show turn critique

Mathematical Behaviors add compute estimate group multiply reduce tabulate bisect count extrapolate integrate number solve tally calculate derive extract interpolate plot square verify check divide graph measure prove subtract

Music Behaviors blow compose harmonize mute pluck sing tap bow finger hum play practice strum whistle clap

Laboratory Science Behaviors apply decrease grow lengthen plant report straighten calibrate demonstrate increase limit prepare reset time conduct dissect insert manipulate remove set transfer connect feed keep operate replace specify weigh convert

General Appearance, Health, and Safety Behaviors button comb eat fill taste untie wear clean cover eliminate go tie unzip zip clear dress empty lace unbutton wait close drink fasten stop uncover wash

Miscellaneous aim discover grind mend relate signify touch attempt distribute guide miss repeat slip try attend do hand offer return slide twist begin drop hang open ride spread type bring end hold pack save stake use buy erase hook pay scratch start vote come expand hunt peel send stock watch complete extend include pin serve store weave consider feel inform position sew strike work correct finish lay present share suggest crease fit lead produce sharpen supply crush fix lend propose shoot support designate flip let provide shorten switch determine get light put shovel take develop give make raise shut tear


Checklist, Performance Quality Rating Scale, or Frequency Rating Scale

Draft Writing Page

Criteria or skills:


Writing Descriptors and Assembling Your Rubric

1. Write the four categories of traits across the top of the table on the following page.

2. Write descriptors for each of the four categories you identified above.

3. Weight each of the 4 traits (yes, they can be weighted equally).

4. Replace the “4, 3, 2, 1” with “Descriptive Terms for Differences in Degree” (or one of your own) to name your levels.


Writing Descriptors and Assembling Your Rubric

Categories of Traits

Weight

_____% _____% _____% _____%

Scale

4 or

Name

Descriptor Descriptor Descriptor Descriptor

3 or

Name


2 or

Name


1 or

Name



Assessment Planning Worksheet (Example)

Performance

Task

Create a collage. Brief

Activity

Description

Students use magazines, photos, posters, and old brochures to create

a collage that communicates a theme about migration. The theme is

something they wish to say or something they believe about

migration. They will use symbols, images, markers, glue, scissors,

and a large sheet of construction paper.

Assessment

Type(s)

Forced Choice Questions □ Checklist □ Performance Quality Rating Scale □ Frequency Rating Scale □ Analytic Rubric ■

Details

(The

questions,

the items,

the criteria,

etc.)

Students will be assessed on the following qualities. (Only the top rubric expectation is written) Craftsmanship: The work was made with few tears, distracting marks, or glue residue.

Use of Color: Complementary colors are evident and create a recognizable pattern.

Use of Shape: There is an overall shape to the work, in addition to internal shapes created by color or

lines.

Theme: The theme being communicated is evident through the use of the symbols/images chosen and

the colors and shapes used.

Objective(s):

Demonstrate use of color, shape, line, and pattern to communicate a theme.


Assessment Planning Worksheet

Performance

Task

Brief

Activity

Description

Assessment

Type(s)

Forced Choice Questions □ Checklist □ Performance Quality Rating Scale □ Frequency Rating Scale □ Analytic Rubric □

Details

(The

questions, the

items, the

criteria, etc.)

Objective(s):

Why use a rubric when a checklist will do?

By Karen L. Erickson

If learning is a journey, think of assessment as a means of transporting students from place to place -‐ from basic knowledge to masterful and complex thinking about subjects where they can synthesize and assimilate ideas. In life, we often choose our mode of transportation or vehicle based on where we are and where we want to go. The same is true for choosing the collection tools we use in assessment. Which tool is appropriate for where students are now? Which tool will best move them forward?

There are a variety of collection tools available for conducting student assessment: checklists, rating scales, qualitative rubrics, and holistic rubrics to name a few. Since selecting the right tool has to do with the learner and her journey to understanding concepts and mastering the skills, rubrics are sometimes, but not always, the answer. How, then, do we determine which tool is most beneficial? To answer this, let’s consider the differences between these types of assessments and the information they return. Let’s begin with a checklist.

Checklists are a simple list of assessment criteria or components that must be present in student work with a space for checking off if the student has accomplished this or not. There is no judgment on the quality of the work. A checklist might look something like this:

Write a Y for Yes; N for No

_______Held the football correctly.

_______Threw a ball 10 yards

_______Kicked the ball over the goal post

Checklists are mostly a yes/no absolute and are best used when introducing basic skills. Some other examples of criteria include whether or not students can define a word; find their own space; put materials away; participate in discussion. Again, it is not about the quality of the work students are doing at this beginning stage. I use a checklist most often in the early grades when I am teaching so many entry level skills and processes. I also use it at any upper grade when I am first introducing a skill or process.

Owner

Stamp

A checklist does not, however, give any indication that the students understand or have mastered the information or skill. When students are beginning to take ownership of the knowledge or skill, I might begin to notice how frequently they use it, get it correct, or choose it for their work. I might also want to communicate a level of judgment about the work. This would be the second stage of moving to mastery and requires a rating scale. Let’s look at two types of rating scales: frequency and performance

A frequency rating scale collects information about how often or in what quantity something happens. It might look something like this:

He kicked the ball for 3 points 3 times 2 times 1 time 0 times

He completed his passes 3 times 2 times 1 time 0 times

He completed his tackles 50% of the time 25% of the time 10% of the time

A performance rating scale might look like this:

You were prepared for the game 5 4 3 2 1

You played with enthusiasm 5 4 3 2 1

Or it might look like this:

You played your position with enthusiasm

Excellent Work Okay Needs Work

How many times have you given or been involved in a professional development workshop and seen an assessment tool that evaluates the workshop with the 5-‐1 scale? But what do all of those numbers mean? Does a 5 from one member of the audience mean the same as a 5 from someone else? Once you know assessment, this becomes frustrating because the responses do little to help you improve for the next professional development program.

How does a student who gets a “Needs Work” know what he has done incorrectly and how to improve it? Yes, there are more scoring choices than a simple checklist and that is a bonus. However, judgments are subjective and your response to a student personally can get in the way of objectivity.

I use rating scales when I see the students moving beyond the simplified application of a skill or process and I want to give them more feedback then a simple yes/no. However, once the

student is ready to move on to demonstrating deeper understanding and is beginning to take charge of their own learning, then I want to move on to a rubric.

A rubric is a tool that has a list of criteria, similar to a checklist, but also contains descriptors in a performance scale which inform the student what different levels of accomplishment look like. A rubric might look like this in our football example. (Did you notice that the examples were all about football? There is a reason for this that you will see shortly.)

Descriptors

Criteria List

Professional

Exceeds

College Sports

Meets

Little League

Not Yet

Passing Your passes were accurate and hit their target every time. They were smooth and spiraled in the air.

Your passes were accurate and hit their target but they wobbled in the air and caused the receiver to stretch.

The passes did not come anywhere near their target.

Kicking Your kicks had follow through and you kept your eye on the ball.

Your kicks had follow through but you took your eye off of the ball.

Your kicks missed the mark because your eye was not on the ball and there was no follow through.

We might even combine a bit of frequency with the descriptors

Blocks You made all clean blocks protecting the quarterback.

You made mostly clean blocks but let the quarterback get sacked.

The quarterback was sacked every time. There were no clean blocks.

The information in a rubric helps the students understand where they are in their own development. When rubrics were developed, it was not for the purpose of giving a grade but for assisting in student learning by providing descriptive feedback. The advantage of rubrics is that students can attain a level they set for themselves and that students can assist in developing them. The challenge of rubrics is writing with clarity. I want to I refer you to “The Art of Words in Crafting Assessments” which can assist you in writing the best possible rubrics.

But what happens when students have mastered the individual criteria and they are beginning to synthesize and assimilate the processes and skills into a unified whole? Then it is time to

move on to a holistic rubric. Holistic rubrics are used when we are no longer looking at individual components but how well and flawlessly components are combined to create a masterful work or performance. These rubrics have a place in the real world and are used quite often when assessing performance of a professional nature. Take a look at this rubric I have been compiling on the Chicago Bears.

Football Rubric based on commentary by Dan Pompei in the Chicago Tribune; combined and edited by Karen L. Erickson – 2009 Bears football season. This 1–10 scale has been compressed here to a 1-‐6 scale for space.

6 5 4 3 2 1

Quarterback Makes 3-‐5 outstanding passes; timing and accuracy impeccable; running when needed; only one interception; manages the game well; takes some risks but does nothing stupid; two long passes -‐ one resulting in a touchdown; one or two stupid plays (e.g. drawing an unsportsmanlike penalty or failing to slide on a short run); moves well; makes nice throws outside of the pocket; makes some poor decisions but ends up with good ones that win the game

Makes the receivers reach for the ball; completes large percentage of passes (77% or more); no running evident; no downfield passes; manages the game well

Nice athleticism and determination; no stupid throws; only one third down conversion; lack of moving the ball into field goal range at half or end of game; failure to achieve touchdowns even when starting a drive on the 30 or 20 yard line of the opposing team; starts out strong and then stops throwing downfield; fumbles one snap at a key moment

Plays it safe; throws the ball away and takes sacks when necessary; conservative play wins the game; makes valuable contributions by running the ball for yardage

Throws three interceptions; fumbles two snaps; underthrown balls to wide open receivers

Red zone interception because of throwing into triple coverage; miscommunications with receivers resulting in interception; sloppy play action fakes; few running plays attempted even though there are good rushers on the team; 3-‐4 almost interceptions and 3-‐4 interceptions in the same game

I noticed in 2009 that Dan Pompei, a sportswriter for the Chicago Tribune, was taking each

position of the Chicago Bears (quarterback, running back, special teams, etc.) and rating them

on a scale of 1-‐10. Following the rating, he would write his justification. I knew immediately

that this was a "rubric" being used in the real world. I wanted to see how consistent he was in

rating the team from week to week, so I created a formal rubric grid and captured (and edited)

his ideas sans names and repetitions. * He was amazingly reliable. At the end of the season I

shared the rubric with him. I have always considered myself quite knowledgeable about

football (my father was a referee and I was raised on the game), but the rubric heightened my

awareness, and gave me new insights into the qualities needed to excel at the game. That is

what a rubric is supposed to do. Even in the arts, we are judged by "invisible" rubrics every time

a critic writes a review. Every reviewer has a rubric in his head against which we are assessed.

If we collected those reviews over time, we could formulate a rubric and gain new insights into

the mind of the author.

Notice that Dan Pompei did not pull out individual criteria, nor should he have, as these are professional football players and their work is judged as a whole, not in parts. We use a holistic rubric to give a broader picture of an entire performance. Holistic rubrics are used to measure the work of the most advanced people in any field of endeavor. Even young students who have outgrown qualitative rubrics can be presented with holistic rubrics (think of young gifted musicians, actors, painters, etc.).

When you are ready to select and develop your collection tools, don’t think you have to use only one at a time. I often have a three criteria rubric on the same page as a four or five criteria checklist or combine a frequency rating scale with a checklist. Fit the assessment to the student and his or her attainment level – sometimes a combination of tools is needed to do the job adequately.

Also keep in mind that all rubrics do not have to have the same number of descriptors. I might be working with two criteria on a rubric that only have three descriptors and two other criteria that have five. Why would they have five? Because the two with five have been studied longer, with more rigor, and the work of the students is more varied.

I have seen people try to write a rubric for a stage in development that only needed a checklist and people writing a checklist when what was needed was a rubric. Remember that learning is a journey. The right vehicle will take the student further down the road to mastery while maintaining a sense of accomplishment by arriving safely and on time at the next stop.

© 2011 Karen L. Erickson Karen can be contacted at [email protected] or through her website www.creativedirections.org * If any of you are football fans and would like a copy of the entire rubric addressing all positions, just send an email to: [email protected]

The Art of Words in Crafting Assessments By Karen L. Erickson As artists we know words can be very powerful. They move us to tears, bring us a smile or two, and paint vivid pictures in our minds. We labor over the right adverb, the right character name, the right phrase that completes a lyrical line, the right dialogue to enhance a mood, the right word to make an audience gasp. It is this same care that dominates our work when we set about writing assessments.

Assessment and Evaluation Speaking of words, I want to begin by defining two important words, assessment and evaluation. Often assessment and evaluation are used interchangeably and in some venues this makes perfect sense. In the world of education, however, which is dominated by the need for accountability and rigor, assessment and evaluation have distinct connotations. * Assessment comes from the Latin “assidere” meaning to “sit by” or “sit beside.” Imagine a master teacher of long ago sitting beside a student learning to play an undocumented song being handed down from one generation to the next. The teacher would be right there listening and watching. When the student had accomplished the piece and was ready to move on to the next, the teacher would present a new song; but if the student was not ready, the teacher would correct, encourage, model, and instruct until the piece was learned. Sounds wonderful – the way teaching and learning should be. However, in a room of 30 students, how does a teacher sit beside each one and make a judgment about his or her learning and understanding? This is where the art of assessment comes in – to create a method where a teacher can sit beside each student in some way. In assessment today, evidence of individual learning is collected and recorded through performance tasks, presentations, papers, dialogue, journals, written work, portfolios, tests, etc. The overall goal of assessment is to improve individual student learning and understanding over time in the content areas. When students have been assessed, a teacher is able to recount who has learned and understands and who does not. The teacher knows who has mastered the piece and who needs more instruction. Evaluation, on the other hand, looks at the bigger broader picture. The root of this word is also Latin, meaning “value.” Here it is about making a value judgment. Evaluation is often reflected in an assigned grade based on subjective analysis. Evaluation reflects the overall quality and talent that a student displays. Teachers make quick evaluations of their class when they see how many students

Owner

Stamp

participate in discussion, the number of hands that get raised following a question, and by looking at the work of a few representative students. Assessment can be used as part of evaluation, but an evaluation cannot be used for assessment. Think of assessment as the collection of ongoing information and evaluation as the overall picture of the value of a student’s work. For example, a classroom of students might participate in a concentration warm-‐up at the beginning of class and you evaluate that they can all concentrate; however, when students present their performance pieces for assessment, some of them show they cannot sustain concentration. They laugh, giggle, and talk to the audience. Through the assessment, you can now see the ability of each individual student to apply the skill of concentration in practice, and you can collect that information to present to the students as feedback. Writing the Assessment Now we turn our attention to how words assist us in collecting data on individual students, painting a picture of how well they are progressing, and providing a vehicle for feedback. Since we have so many students to sit beside, we often do this in writing (though verbal feedback can also be powerful). One of the best collection tools for giving detailed feedback is a rubric, which can be used for any type of student performance or product. The rubric has two parts: criteria and descriptors indicating different levels of accomplishment. When writing a rubric, the preciseness of our words becomes essential, especially if we want to use it for reliable assessment purposes and apply it in an objective and fair way.

Criteria Let’s start with the criteria. This is the list of things you want to measure. First, select criteria that are highly important to the subject area. Often I have seen teachers select criteria (concepts and skills) that are easy to assess but are not necessarily the most important areas of learning to assess. For example, in music, I saw a teacher labor over taking each child aside to see if they could identify a loud sound from a soft sound. This took two days of valuable class time. Would this have been better to be evaluated rather than assessed? Was it important to assess all students to see if they could identify loud and soft? Or is it more important to see if each student can identify or analyze the type of sounds that contribute to creating a mood? It is for each teacher to think deeply about the criteria being placed on a rubric and not necessarily go for what is easy. It is to consider what is most important. Second, is this the criteria you are really assessing? I read a rubric the other day that had creativity listed as one of the criteria for an art project. Now, I will admit that creativity is something that should be taught with rigor through our art forms. It is essential content. But the descriptors for creativity clearly indicated the teacher was looking at neatness, use of materials, and craftsmanship -‐ not creativity. It made me wonder then if creativity as a skill or concept had even been taught. Third, make sure you have explicitly taught that criteria before you assess it. Does more need to be said here?

Fourth, have the students suggest the criteria. If you are wondering if you have fairly selected criteria and adequately taught them, have the students list the things they think should be included in the assessment. This will provide you with a clear snapshot of what came through in your teaching.

Painting the Picture: Selecting the Right Adverb or Adjective Now let’s turn out attention to the descriptors: those little boxes on a rubric that describe what exceeds, meets, does not meet, and absent looks like (or Academy Award Winner, Lead Actor, Understudy, Back to Acting School as I like to use). There are so many things to consider here. A descriptor should be:

• written in student language • written so a person outside of your art form could use it – no jargon • free of fuzzy words • focused and not overly compounded • written so frequency words make sense

Let’s take these factors one at a time: It should be written in student language. A rubric is for the student. It sets out what the student should be doing, how they should be doing it, and what it looks like in best practice. It is there so the students can see a target, understand the target, aim as high as they can, and be able to fairly determine where they have landed until they get to try again. So it should be written in grade level appropriate language and given to the students throughout instruction. It should be written so a person outside of the art form can use it. Assess your rubric with another adult sitting beside you. When he or she uses the rubric is the result the same? Check the rubric for jargon from the art form that only an insider would know. Now, if this is a word you use with the students daily and is part of their vocabulary, make an exception. You will diminish the universality of the rubric, but it will still be fair in your instance. It should be free of fuzzy words. This is crucial. I see many words used in rubrics that make me wonder -‐ words like clearly, appropriately, well, etc. If I read, “He sang the song appropriately” in a rubric, I would ask, “What does that mean, ‘appropriately’?” If the person answers, “It was loud enough for everyone to hear, there were no pitch problems, and the words were all memorized,” I would respond by saying, “Well, write that in your descriptor – that makes sense.” This is another reason to have someone sit beside us as we polish our rubrics, because even the best of us slip in these subjective words that have no substance. We don’t want to tell a child, “You didn’t sing appropriately,” and have him go away not being aware of what was incorrect and how to improve it. The whole notion behind assessment is to improve learning. To do this, we must be clear, concise, complete, and as minimalistic as possible. It should be focused and not overly compounded. Let’s return to our music example which is now free of fuzzy words. Once we have determined what we mean by “appropriately” we now have a list of skills -‐ loudness, on pitch, memorized – within the descriptor. The next step is to determine if each of these should be their own criteria, with their own descriptor, or if they should continue to be grouped

together. Think this through. What if the student is loud and on pitch, but misses a word or two? Does this move her down to the next level? Do all three of these items carry the same weight or value within the assessment? Splitting or lumping is a major decision. It should written so frequency words make sense First, congratulations for considering a rubric when a frequency rating scale would be so much easier. By frequency words I mean: all, some, none, most, few, never, sometime, 3 times, 50%, etc. A frequency rating scale uses frequency words but no descriptors. Sometimes people like to combine the two. This should be done with care as sometimes it creates fuzziness. For instance, sticking with our music example, let’s say we write the following descriptors for memorization: remembered all of the words to the song; remembered some of the words to the song; remembered a few of the words to the song; remembered none of the words to the song. As a student, my questions would be, “What is the difference between some and few? Couldn’t some be a few? Could a few be some?”

Words are so important when crafting an assessment. Don’t be afraid to elicit support and work as a team with others. The writing should be specific and clear to paint a picture for the student of what his work, project, or performance should look like EXPLICITLY. We want students to understand and to be able to demonstrate their understanding.

Remember… assessment needs the right words, choosing them is an art.

© 2011 Karen L. Erickson Karen can be contacted at [email protected] or through her website www.creativedirections.org Biography: Karen L. Erickson, a national consultant in arts education, professional artist and Executive Director of Creative Directions, provides training in curriculum, arts integration assessment, whole school change, playwriting, directing, drama education, and language arts nationally and internationally. Erickson is a Workshop Leader and Seminar Presenter for the Kennedy Center’s professional development programs. Erickson is a certified teacher of theater, language arts, and speech communications K-‐12. Author of seven drama education books, she co-‐authored the Illinois Learning Standards for Fine Arts, Chicago Arts Standards, and the Integrated Curriculum Arts Project (ICAP). Erickson served as Artistic Director of Trinity Square Ensemble Theater in Evanston and worked at the Goodman Theatre as Assistant to Tennessee Williams. Erickson continues her work as a playwright and stage director having written fifteen plays for youth and adults produced across the United States. _________________________________________________________________________________ * The Illinois State Board of Education in “Assessment Handbook: A Guide to Assessing Illinois Students,” says “The terms test, assessment, and evaluation are frequently used interchangeably, but, in fact, have important differences.” “Assessment is more encompassing. Assessment provides more comprehensive data on student performance through several administrations of test batteries or through various other data gathering approaches.” “Evaluation, as the word itself suggests, refers to making a value judgment about the implications of assessment data. While assessment involves obtaining performance data through a variety of means, evaluation goes a step further – interpreting the data from an informed perspective.” “Evaluation produces value judgments about the results provided through assessment.”

Documents

Reviewing UBD and Moving Deeper into Assessment€¦ · ©2012, Karen L. Erickson. Do not duplicate without permission. Reviewing UBD and Moving Deeper into Assessment with Karen