Best Practices In College Teaching : Designing Effective Rubrics

Best Practices In College Teaching: Designing Effective Rubrics

Debra Dunlap Runshe

Instructional Development SpecialistUniversity Information Technology Services - Learning Technologies

Indiana University – Purdue University Indianapolis

Have you ever had a student look at you like this?

Have you ever felt like this?

Do you need a rubric?

If you agree with any of these statements a rubric may be for you…

You are getting carpal tunnel syndrome from writing the same comments on almost every student paper.

You have graded all your papers and worry that the last ones were graded slightly differently from the first ones.

You’ve sometimes been disappointed by whole assignments because all or most of your class turned out to be unaware of academic expectations so basic that you neglected to mention them (e.g., the need for citations or page numbers).

(Stevens & Levi, 2005)

Webinar Objectives

By the end of this webinar, you will be able to:articulate how rubrics can be

useful.describe characteristics of a

rubric.distinguish between analytic

and holistic rubrics.design a rubric.

Where are you now?

Do you use rubrics to assess student work? If so, how?

What do you already know about rubrics?

What are some things you want to learn about using rubrics to assess student work?

Definition of a Rubric

Rubrics are criterion-referenced rules for assessing student performance holistically or analytically (on different dimensions).

A scoring tool that lays out the specific expectations for an assignment, providing a detailed description of what constitutes acceptable or unacceptable levels of performance.


Parts of a Rubric

Task description

A scale

Dimensions of the assignment

Descriptions of performance level


Analytic vs. Holistic Rubrics

Analytic rubric: Common when evaluating independent dimensions or components of student work.

Holistic rubric: Common when the assessed criteria are considered in combination and when quality is judged broadly.

Holistic Rubric

Task Description

Scale Level 1

•Dimension 1 description•Dimension 2 description•Dimension 3 description

ScaleLevel 2


ScaleLevel 3


ScaleLevel 4


ScaleLevel 5


Holistic Rubric

A•Insightful development and mature style•Cogent analysis of or response to the text•Uses sophisticated sentences effectively

B•Clearly competent•Thoughtful analysis of or response to the text•Less fluent and complex style than 6, but chooses words accurately, varies sentences effectively

C•Satisfactory•Adequate analysis of response to the text•Usually chooses words of sufficient precision, sentences of reasonable variety

D•Unsatisfactory in one or more ways•May analyze or respond to text illogically•Frequently imprecise word choice and little sentence variety

F•Serious weaknesses, of several kinds•Simplistic, inappropriate, or incoherent analysis of or response to text•Inaccurate word choice, monotonous or fragmented sentence structure

Article Review

Analytic Rubric

Scale Level 1

Scale Level 2

Scale Level 3

Scale Level 4

Dimension 1

Dimension 2

Dimension 3

Task Description

Analytic Rubric

Exemplary Competent DevelopingContent Full understanding

of topicGood Understanding of parts of topic

Does not seem to understand topic

Stays on topic Stays on topic 100-95% of the time

Stays on topic 94-75-% of the time

It was hard to tell what the topic was

Preparedness Completely prepared and has obviously rehearsed

Somewhat prepared, but it is clear that rehearsal was lacking

Does not seem at all prepared to present

Eye contact Establishes eye contact with everyone in the room

Sometimes establishes eye contact

Does not look at people during the presentation

Oral Presentation

Constructing a Rubric

A rubric is a protocol for grading based on:

critical componentsscoring scale

So, what is worth grading on (the components)?

What is worth grading on?

Comprehensive literature review

Clarity of rational for chosen research design including importance to field

Understanding of methodology to be used

Includes and addresses potential limitations and implications for practice

Proper references to texts, other resources

Organization, conformity to format

Precision of measurement, quality of data

Clarity of explanations, expression

Strength/tightness of arguments

Grammar and mechanicsWriting styleUse of APA Style

Constructing a Rubric

Four Important Steps

1. Reflection

2. Listing of Objectives

3. Grouping and Labeling

4. Application of Scales


Step 1: Reflection

Eight questions to consider:

1. Why did you create assignment?2. Have you given this or similar assignment before?3. How does the assignment relate to the rest of the

course?4. What skills do students need for successful

completion?5. What exactly is the task assigned?

6. What evidence can students provide to show they have successfully completed the assignment?

7. What does an exemplary product look like?8. What does the worst example of a product look like?


Step 1: Reflection

Objectives

Activities Assessment

Step 2: Listing of Objectives

What are the specific learning objectives for this assignment?

What is the highest level of performance you expect for each learning goal?

Step 3: Grouping and Labeling

Dimensions of an Academic Research Proposal

Research Question

Literature Review

Methodology

Limitations and Significances

Transitions

Grammar and Style

Step 4: Application of Scales

Exemplary, proficient, marginal, unacceptableAdvanced, intermediate high, intermediate, noviceDistinguished, proficient, intermediate, noviceAccomplished, average, developing, beginningOutstanding, very good, good, poor, unsatisfactoryA, B, C, D, FSatisfactory, unsatisfactory

Scoring Scale Example

“Elegance of Argument” component:

5 Original and clearly stated thesis, persuasive, well-organized, imaginative use of source material4 Clearly stated thesis, good use of sources, well

organized3 Facts straight with reasonable explanation of the subject under consideration2 Poorly stated thesis, inadequate survey of available sources, poor organization1 No awareness of argument or complexity

Reliability and Validity

Reliability - Does it get the same results consistently?

Would two experts from the same discipline score student demonstration the same with the rubric?

Validity – Does it measure what it claims to measure?

Would two experts from the same discipline consider what the student is asked to demonstrate an example of what you want to measure?

Let’s Create a Rubric

Think about building a rubric for buying a house.Identify the components that are critical in

completing this task.Write the descriptive

levels for the

components.

Step 1: Reflection

Step 2: Listing of Objectives

Step 3: Grouping and Labeling

Step 4: Application of Scales

DEVELOPING

UNSATISFACTORY

SATISFACTORY

ACCOMPLISHED

PROFICIENT

Advantages of a Clear Rubric

Provides timely feedbackPrepares students to use detailed feedbackEncourages critical thinkingFacilitates communications with othersHelps refine teaching methodsLevels the playing field


Rubrics Help Students to Self-assess

Provide examples of work corresponding to different levels

Provide coaching on components of quality before the whole assignment is undertaken

Have students use the rubric to assess each other’s practice assignments and develop norms of fairness

Have students use the rubric to assess their own work and provide feedback on their self-assessment

Other Considerations

Research other rubrics

Peer evaluation

Revise, revise, revise

eTools for Rubric Construction

RubiStar:

http://rubistar.4teachers.org

iRubric:

http://www.rcampus.com/indexrubric.cfm

AAC&U’s VALUE Rubrics

(Retrieved from http://www.aacu.org/value/rubrics/index_p.cfm?CFID=41453385&CFTOKEN=58615771 August 8, 2012)

http://www.aacu.org/value/rubrics/index_p.cfm?CFID=41453385&CFTOKEN=58615771

http://www.aacu.org/value/rubrics/index_p.cfm?CFID=41453385&CFTOKEN=58615771

VALUE Rubrics

Intellectual and Practical SkillsInquiry and analysisCritical thinking Creative thinking Written communication Oral communication Reading Quantitative literacy Information literacy Teamwork Problem solving

Learning Outcomes for the development of VALUE Rubrics

Personal and Social ResponsibilityCivic knowledge and engagement-

local and global Intercultural knowledge and

competence Ethical reasoning Foundations and skills for lifelong

learning

Integrative and Applied Learning Integrative and applied learning

(Retrieved from http://www.aacu.org/value/abouttherubrics.cfm August 8, 2012)

http://www.aacu.org/value/abouttherubrics.cfm

http://www.aacu.org/value/abouttherubrics.cfm

Summary

During this webinar, we:

discussed how rubrics can be useful.described characteristics of a rubric. distinguished between analytic and holistic rubrics. designed a rubric.

Why use rubrics?

So students look like this …

Thank You for Your Participation!

Debra Dunlap Runshe, Instructional Development SpecialistUniversity Information Technology Services – Learning Technologies

Indiana University-Purdue University IndianapolisInformation Technology and Communications Complex (IT 342H)535 West Michigan Street, Indianapolis, IN 46202

Phone: 317-278-0589 Email: [email protected]

References and Resources

ALTEC at University of Kansas. RubiStar. Web site: http://rubistar.4teachers.org

Carnegie Mellon, Eberly Center for Teaching Excellence. Grading and performance rubrics. Retrieved June 11, 2010 from http://www.cmu.edu/teaching//designteach/teach/rubrics.html

Kansas State University, Office of Assessment. Measures, rubrics, & tools for assessing student learning outcomes. Web site: http://www.k-state.edu/assessment/plans/measures/samples/index.htm

McGonigal, K. (2006, spring). Getting more “teaching” out of “testing” and “grading." Speaking of Teaching, 15, 2. Retrieved June 11, 2010 from http://ctl.stanford.edu/Tomprof/postings/738.html

Moskal, B. M. (2003). Recommendations for developing classroom performance assessments and scoring rubrics [Electronic version]. Practical Assessment, Research & Evaluation, 8(14). Retrieved June 11, 2010 from http://pareonline.net/getvn.asp?v=8&n=14

Moskal, B. M. (2000). Scoring rubrics: what, when and how? [Electronic version]. Practical Assessment, Research & Evaluation, 7(3). Retrieved June 11, 2010 from http://PAREonline.net/getvn.asp?v=7&n=3

http://rubistar.4teachers.org/

http://www.cmu.edu/teaching/designteach/teach/rubrics.html

http://www.k-state.edu/assessment/plans/measures/samples/index.htm

http://ctl.stanford.edu/Tomprof/postings/738.html

http://pareonline.net/getvn.asp?v=8&n=14

http://pareonline.net/getvn.asp?v=7&n=3

References and Resources

Palomba, C.A., & Banta, T.W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass.

Quinlan, A. M. (2006). A complete guide to rubrics: Assessment made easy for teachers, K-college. Lanham, MD: Rowman & Littlefield.

Reazon System, Inc. iRubric. Web site: http://www.rcampus.com/indexrubric.cfm

rSmart, a Sakai Commercial Affiliate. Resources for teaching and learning. Web site: http://openedpractices.org/resources

Simkins, M. (1999). Designing great rubrics. Technology & Learning, 20 (1), 23-24, 28-30.

Stevens, D. D. & Levi, A. J. (2005). Introduction to rubrics. Sterling, VA: Stylus.

Suskie, L. (2009). Assessing student learning: A common sense guide. (2nd ed.). San Francisco: Jossey-Bass.

TLT Group. Rubrics. Web site: http://www.tltgroup.org/resources/Rubrics.htm

Walvoord, B.E. (2010). Effective grading: A tool for learning and assessment. San Francisco: Jossey-Bass.


http://openedpractices.org/resources

http://www.tltgroup.org/resources/Rubrics.htm

What are rubrics? A rubric is a criterion-referenced scoring tool for assessment linked to learning objectives that is used to assess a student's performance. Rubrics allow for standardized evaluation according to specified criteria, making grading simpler and more transparent.

What benefits do rubrics offer, and to whom?Rubrics benefit both the instructor and the students. They are used both to guide student learning and to assess student learning outcomes.

Creating rubricsSteps in creating rubrics include: 1) articulate the objective(s) of the assignment; 2) identify criteria to be evaluated; 3) determine the levels of performance across the criteria; 4) describe the performance at the various levels for each criterion.

Resources for rubric creationAAC&U’s VALUE rubrics from www.aacu.org/value/metarubrics.cfm Rubistar http://rubistar.4teachers.org iRubric http://www.rcampus.com/indexrubric.cfm

Quick Guide to Rubrics

http://www.aacu.org/value/metarubrics.cfm

http://rubistar.4teachers.org/


Rubric Template(Describe here the task or performance that this rubric is designed to evaluate.)

Beginning1

Developing2

Accomplished3

Exemplary4

Score

Stated Objective or Performance

Description of identifiable performance characteristics

reflecting a beginning level of performance.

Description of identifiable performance characteristics reflecting development and

movement toward mastery of performance.


reflecting mastery of performance.

Description of identifiable performance characteristics reflecting the highest level of

performance.









performance.









performance.









performance.

Rubric Title (highlight and replace with your title)

(Description of task or performance - highlight and replace with your description.)

Score

Documents

Best Practices In College Teaching : Designing Effective Rubrics