Rubrics in Student Affairs

Carrie Zelna, Ph.D. Director of Student Affairs Research and Assessment

NCSUcarrie_zelna@ncsu.edu

Session Outcomes

1. Participants will be able to define the term ‘rubric.’

2. Participants will be able to describe the types and styles of rubrics.

3. Participants will be able to articulate how rubrics can be used to measure learning/development outcomes.

4. Participants will be able to articulate how data from rubrics can be used to make decisions.

Rubric: Definition and Purpose

Rubric: “a scoring tool that lays out the specific expectations for an assignment” (Stevens & Levi, 2005, p. 3)

It is a way of organizing criteria to systematically determine if the outcome is met based on data gathered through papers, observation, document analysis, or some other appropriate method.

Rubric: Types

Types of Rubrics: Analytic (specific feedback along several dimensions)

Holistic (single score)

General (criteria is general across tasks)

Task Specific (unique to a task)From: http://its.monmouth.edu/facultyresourcecenter/rubrics.htm April

11, 2007

Components of a Rubric

Note** You would have already determined the outcome(s) to be measured and the best method of

collecting data such as observation, case study, reflection paper, etc.

Adapted from Stevens, D.D., & Levi, A.J. (2005). Introduction to Rubrics. Stylus Publishing, LLC Sterling, VA.

Part One:

Carefully determine the implementation of the method for gathering the information.

Who: Sample or entire population

When: Look at the calendar

How: Carefully consider questions asked or tasks to be performed

Part Two:

Determine the dimensions of the rubric-what do you want to measure more specifically?

  Scale Level 1 Scale Level 2 Scale Level 3 Scale Level 4

Dimension 1        

Dimension 2        

Dimension 3        

Stevens, D.D., & Levi, A.J. (2005). Introduction to Rubrics. Stylus Publishing, LLC Sterling, VA.

Defining DimensionsThe dimensions are the criteria for your outcome.

Some examples are:Leadership: communication, decision making, motivation, etc.Sportsmanship: cooperate with officials, remain calm when interacting with opposite team, no foul language, etc.Active Listening Skills: Sits leaning slightly forward, makes eye contact, nods, asks open ended questions, etc.Problem Solving Skills: Identifies the problem, identifies the available options, able to recognize the consequences for each option, etc.

Part Three:

Determine the size of the scale to be used and the appropriate labels based on the criteriaThe scale generally is across the top of the rubricYes/no check list or scale of 3-5 (although some recommend even numbers)

  Scale Level 1 Scale Level 2 Scale Level 3 Scale Level 4

Dimension 1        

Dimension 2        

Dimension 3        

Stevens, D.D., & Levi, A.J. (2005). Introduction to Rubrics. Stylus Publishing, LLC Sterling, VA.

Scale Examples (Style)

Yes, No

Beginner, Intermediate, Advanced

Exemplary, Accomplished, Developing, Beginning

High mastery, Average mastery, Low Mastery

Part Four:

Describe each dimension in some detail and/or….

Describe what it should look like based on each of the levels of the scale you have chosen.

Sample Group Project Rubric

Teacher Name: Dr. Elling

Student Name:     ________________________________________

CATEGORY 4 3 2 1Preparedness Brings needed

materials to class and is always ready to work.

Almost always brings needed materials to class and is ready to work.

Almost always brings needed materials but sometimes needs to settle down and get to work

Often forgets needed materials or is rarely ready to get to work.

Attitude Never is publicly critical of the project or the work of others. Always has a positive attitude about the task(s).

Rarely is publicly critical of the project or the work of others. Often has a positive attitude about the task(s).

Occasionally is publicly critical of the project or the work of other members of the group. Usually has a positive attitude

Often is publicly critical of the project or the work of other members of the group. Often has a negative attitude

Working with Others Almost always listens to, shares with, and supports the efforts of others. Tries to keep people working well together.

Usually listens to, shares, with, and supports the efforts of others. Does not cause "waves" in the group.

Often listens to, shares with, and supports the efforts of others, but sometimes is not a good team member.

Rarely listens to, shares with, and supports the efforts of others. Often is not a good team player.

Focus on the task Consistently stays focused on the task and what needs to be done. Very self-directed.

Focuses on the task and what needs to be done most of the time. Other group members can count on this person.

Focuses on the task and what needs to be done some of the time. Other group members must sometimes nag,

Rarely focuses on the task and what needs to be done. Lets others do the work.

Monitors Group Effectiveness

Routinely monitors the effectiveness of the group, and makes suggestions to make it more effective.

Routinely monitors the effectiveness of the group and works to make the group more effective.

Occasionally monitors the effectiveness of the group and works to make the group more effective.

Rarely monitors the effectiveness of the group and does not work to make it more effective.

Quality of Work Provides work of the highest quality.

Provides high quality work.

Provides work that occasionally needs to be checked/redone by other group members to ensure quality.

Provides work that usually needs to be checked/redone by others to ensure quality.

Collaborative Work Skills : Group Project

Review the Entire Process

Once you have all the ‘Parts,’ review the entire process from the beginning: Outcome, method and questions/situation, dimensions, scale and descriptions.

Will you be able to gather the necessary data to determine where they fit on the scale for each dimension?

Testing Your Rubric

Use a Metarubric to review your work (see pg. 94 of Stevens and Levi [2005] for example)

Peer review- ask one of your peers to review the rubric and provide feedback on content

Student review-ask a student to do so as well if appropriate

Test with students - use student work or observations to test the rubric

Revise as needed

Test againMultiple raters – norm with other raters if appropriate

Yes / No Checklist: SAS inSchool

http://www.SASinSchool.com

Scoring

Individual vs. Aggregate Scores

Average Score By Dimension and TotalTotal Score: Total scores may be reviewed to get a big picture

Dimension: Dimension scores to look for patterns

Frequency DistributionsScale: Frequencies by scale to get a clearer understanding of the data

Scoring the Data

ID# Class Age GenderPaper Length Total

Separation/ Objectivity Dissonance

Understanding/ Change in

PerspectiveSelf-

Perception ResolutionApplication/ Verification totals

A FR 19 F 5 18 3 3 3 3 3 3 18B SR 21 M 3 17 3 3 3 3 3 2 17C FR 18 F 7 16 3 3 3 2 2 3 16D SR 21 M 5 16 3 3 3 2 3 2 16E SO 19 F 9 15 2 3 3 2 2 3 15F FR 18 M 3 14 3 3 3 2 3 0 14G SO 20 M 3 14 3 3 3 0 3 2 14H SO 19 M 5 13 2 2 3 2 2 2 13I FR 18 M 8 13 3 3 3 2 2 0 13J JR 20 F 5 13 2 2 2 2 3 2 13K SO 20 M 5 13 3 3 2 2 2 1 13L FR 18 M 7 13 2 3 2 2 2 2 13M JR 20 F 3 11 3 3 3 0 2 0 11N FR 18 F 5 10 2 2 2 2 2 0 10O SO 22 M 4 10 2 3 2 2 2 0 11P FR 18 F 6 10 2 3 1 2 1 1 10Q FR 19 M 9 9 2 2 1 2 1 1 9R FR 18 M 3 9 2 3 2 1 1 0 9S FR 18 M 15 7 2 1 1 1 1 1 7T SO 20 F 4 7 1 2 0 1 2 1 7

Average Score 2.526315789 2.789473684 2.368421053 1.84210526 2.21052632 1.368421053 13.10526316

Frequencies Scale: 3 Scale:2 Scale: 1 Scale: 0

Separation/ Objectivity 9 10 1 2

Dissonance 14 5 1 2

Understanding/ Change in Perspective 10 6 3 3

Self-Perception 2 13 3 4

Resolution 6 10 4 2

Application/ Verification 3 6 5 8

Frequencies

By Jonathan Schnyer at University Virginia

Insight Rubric for NCSU Judicial System Reflection Paper

Evaluating Student

Presentations

Developed by Information Technology Evaluation Services, NC Department of Public Instruction

1 2 3 4 Total

Organization

Audience cannot understand presentation because there is no sequence of information.

Audience has difficulty following presentation because student jumps around.

Student presents information in logical sequence which audience can follow.

Student presents information in logical, interesting sequence which audience can follow.

Subject Knowledge

Student does not have grasp of information; student cannot answer questions about subject.

Student is uncomfortable with information and is able to answer only rudimentary questions.

Student is at ease with expected answers to all questions, but fails to elaborate.

Student demonstrates full knowledge (more than required) by answering all class questions with explanations and elaboration.

Graphics Student uses superfluous graphics or no graphics

Student occasionally uses graphics that rarely support text and presentation.

Student's graphics relate to text and presentation.

Student's graphics explain and reinforce screen text and presentation.

Mechanics Student's presentation has four or more spelling errors and/or grammatical errors.

Presentation has three misspellings and/or grammatical errors.

Presentation has no more than two misspellings and/or grammatical errors.

Presentation has no misspellings or grammatical errors.

Eye Contact Student reads all of report with no eye contact.

Student occasionally uses eye contact, but still reads most of report.

Student maintains eye contact most of the time but frequently returns to notes.

Student maintains eye contact with audience, seldom returning to notes.

Elocution

Student mumbles, incorrectly pronounces terms, and speaks too quietly for students in the back of class to hear.

Student's voice is low. Student incorrectly pronounces terms. Audience members have difficulty hearing presentation.

Student's voice is clear. Student pronounces most words correctly. Most audience members can hear presentation.

Student uses a clear voice and correct, precise pronunciation of terms so that all audience members can hear presentation.

Total Points:

Name: Caroline McCullen http://www.ncsu.edu/midlink/rub.pres.html

Apply critical thinking skills to solve problems, make informed decisions, and interpret events. http://academic.pgcc.edu/~wpeirce/MCCCTR/Designingrubricsassessingthinking.html Designing Rubrics for Assessing Higher Order Thinking by William Peirce

Rubric Component

4 3 2 1

Identifies and summarizes the problem/question at issue.

Accurately identifies the problem/question and provides a well-developed summary.

Accurately identifies the problem/question and provides a brief summary.

Identifies the problem/ question and provides a poor summary or identifies an inappropriate problem/question.

Does not identify or summarize the problem/question accurately if at all.

Identifies and assesses the quality of supporting data/evidence

Provides a well-developed examination of the evidence and questions its accuracy, relevance, and completeness. Clearly distinguishes between fact and opinion.

Examines evidence and questions the quality. Distinguishes between fact and opinion.

Merely repeats information provided. Does not justify position or distinguish between fact and opinion.

Does not identify or assess the quality of supporting evidence.

Identifies and considers the influence of the context* on the issue

Accurately identifies and provides a well-developed explanation of contextual issues with a clear sense of scope.

Accurately identifies and provides an explanation of potential contextual issues.

Does not explain contextual issues; provides inaccurate information; or merely provides a list.

Does not identify or consider any contextual issues.

Demonstrates higher level thinking by interpreting the author’s meaning or the potential bias

Accurately identifies the author’s meaning and/or potential bias and provides a well-developed explanation.

Accurately identifies meaning and/or bias and provides a brief explanation.

Does not explain, provides inaccurate information, or merely lists potential bias or inferred meanings.

Identifies and evaluates conclusions, implications, and consequences

Accurately identifies conclusions, implications, and consequences with a well-developed explanation. Provides an objective

Accurately identifies conclusions, implications, and consequences with a brief evaluative summary.

Does not explain, provides inaccurate information, or merely provides a list of ideas; or only discusses one area.

Does not identify or evaluate any conclusions, implications or consequences.

Service Learning Rubric

Description: Service-Learning is a teaching method that combines academic instruction, meaningful service, and critical reflective thinking to enhance students learning and civic responsibility. Use this rubric to evaluate your progress before, during and after your service-learning project. This rubric was modified from the World Wise Schools Website.

4 3 2 1

Meets actual needs Determined by current research conducted or discovered by students with teacher assistance where appropriate

Determined by past research discovered by students with teacher assistance where appropriate

Determined by making a guess at what community needs may be

Community needs secondary to what a project teacher wants to do; project considers only student needs or desires

Coordinated w/community Active, direct collaboration with community by the teacher and/or student

Community members act as consultants in the project development

Community members are informed of the project directly

Community members are coincidentally informed or not knowledgeable at all

Integrated academically Service-learning as instructional strategy with content/service components integrated

Service-learning as a teaching technique with content/service components concurrent

Service-learning part of curriculum but sketchy connections, with emphasis on service

Service-learning supplemental to curriculum, in essence just a service project or good deed

Facilitates reflection Students think, share, produce reflective products individually and as group members

Students think, share, produce group reflection only

Students share with no individual reflective projects

Ran out of time for a true reflection; just provided a summary of events

Uses academics in real world All students have direct application of new skill or knowledge in community service

All students have some active application of new skill or knowledge

Some students more involved than others or little community service involvement

Skill knowledge used mostly in the classroom; no active community service experience

Develops caring Reflections show affective growth regarding self in community and the importance of service

Reflections show generic growth regarding the importance of community service

Reflections restricted to pros and cons of particular service project regarding the community

Reflections limited to self-centered pros and cons of the service project

Improves quality of life Facilitate change or insight; help alleviate a suffering; solve a problem; meet a need or address an issue

Changes enhance an already good community situation

Changes mainly decorative, but new and unique benefits realized in community

Changes mainly decorative, but limited community benefit, or are not new and unique

http://www.uen.org/Rubric/rubric.cgi?rubric_id=359

Rubistar – A Free Web-Based Rubric Template Generatorhttp://rubistar.4teachers.org/index.php

References/Resources

Huba and Freed (2000) Learner-Centered Assessment on College Campuses.

Stevens and Antonia (2005) book Introduction to Rubrics. Stylus Publishing, LLC.http://its.monmouth.edu/facultyresourcecenter/rubrics.htm

SAS inSchool http://www.SASinSchool.com