Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Paula O’Neill, MEd, EdDAssociate Dean for Educational Research
Professor, Diagnostic SciencesThe University of Texas Dental Branch at
1
Rationale and role in using rubrics in dental education
Describe best practices for constructing rubrics for dental education.
A model for evaluating rubrics
2
A Rubric shows how learners will be assessed and/or graded.
A rubric provides a clear guide as to how ‘what learners do’ in a course will be assessed.
3
Scoring tool◦ Specific expectations
Criteria and standards◦ Linked to learning objectives
Consistent assessment criteria
Shared with students◦ Work towards success
Can be either grading or performance rubrics
4
1. You give careful descriptions of assignments, but students are still asking you many questions about the assignment;
2. You have been disappointed by the outcomes of assignments because your students seem to be unaware of your academic expectations;
3. You want students to be able to solve complex problems, yet students fail to understand those expectations;
4. You receive feedback that “your grading practices are not be calibrated with other faculty.”
5
Discussion participation Laboratory work Clinical competencies Group work Oral presentations Essay exams Research papers Student portfolios
6
Reduced grading time Memory of assignments,
competencies, etc at the “front of the mind.”
Provides more specific intentions and expectations
Valuable teaching and learning tool◦ More aware of teaching styles and strategies
Problem solving and critical thinking Classroom dialogue between faculty &
students
7
1. Reflecting
2. Listing criteria
3. Grouping and labeling
4. Application
Other considerations◦ Student involvement?◦ Evaluation of rubric effectiveness
8
What do you want from the students? Why did you create the assignment/test? How does this assignment related to other
content? What skills do students need to have/develop to
“successfully” complete the assignment? What is the exact task assigned? What evidence will students provide to
demonstrate accomplishment? Your highest expectation-Excellence- for
performance is? What is the Poorest fulfillment of the assignment?
9
Specific Learning Objectives
Objectives will vary
10
Organization of Steps 1 and 2
Group similar expectations together in what could become ◦ Dimensions
11
Transfer lists and groups to rubric grid Labels become Dimensions◦ Placement: left column of rubric
Learning tasks/objectives ◦ Become descriptions of highest performance level
Exemplary Competent Developing
Individualpresentations
Spoke clearly, slowly and loudly modulating voice and quality
Was understood but mumbled, spoke too fast or slow, whispered, shouted, not compromised
Mumbled, spoke too softly-loudly, intelligibility was compromised
Group Work Each member given equal opportunity to shine
Presentations followed logically but connections unclear
Little logic evident, connections between presenters unclear
12
1. Task description measuring specific objective
◦ Performance, behavior or quality
2. Scale/range indicating levels of competency, achievement
3. Dimensions of the assignment/task
4. Description of what constitutes each level of performance
KEY COMPONENTS OF DEVELOPMENT
◦ Determine level of student involvement
◦ Pretest/pilot rubric
13
Framed by faculty member◦ Focused upon “performance” expectation◦ Specific task or overall behavior
SCALE Level 1 SCALE Level 2 SCALE Level 3
Dimension 1
Dimension 2
Dimension 3
Dimension 4
14
Description of how well or poorly any expected task has been performed
Start with 3 levelsExcellent Competent Needs Work
Dimension 1
Dimension 2
Dimension 3
Dimension 4
15
Lay out the parts of the rubric
Start with Task Analyses
Provide clarification for the students of how the task for them to accomplish can be◦ Broken into components◦ Which of the components is most important i.e.., facts, research techniques, skill competency◦ Description of how much weight will be given to
each of the components
16
Should not include any description of the QUALITY of the performance◦ “Good, outstanding, poor etc.
Together with good descriptions, the dimension provide detailed feedback on specific parts of an assignment◦ How well or poorly accomplished
17
Description of highest-lowest level of performance
Excellent Competent Needs Work
Knowledge40%/40 points
Thinking30%/30 points
Communication20%/20 points
Presentationskills 10%/10 Points
18
Student’s highest level of performance Three levels of performance more useful
Excellent Competent Needs Work
Knowledge40%/40 points
The case report demonstrates a depth of understanding
Report generally accurate with only minor inaccuracies
Report included little relevant or accurate info about patient
Thinking30%/30 points
The report centered about high level of conceptual ability
Analyticalstructure but analysis not always linked
No analyticalstructure evident
Communication20%/20 points
Case presented effectively & linked to lab reports
Case presented well but questions not all answered
Case as presented confusing in what is communicated
Presentationskills 10%/10 Points
Student speaks clearly, using appropriate terminology
Student speaks clearly, but terminology not always used
Student does not speak clearly nor use correct terminology
19
20
Presentation model — most common◦ Faculty member does all the work, makes decisions◦ Sets expectations, weight, scale, levels and
fulfillmentRubric ConstructionModel
Stage 1:Reflecting
Stage 2: Listing
Stage 3: Groupingand Labeling
Stage 4: Application
1.Presentation
Facultymember
Faculty member
Faculty Member
Faculty member & students who ask questions
21
Actually present to class and encourage feedback for revision purposes◦ Small group activity
Rubric ConstructionModel
Stage 1:Reflecting
Stage 2: Listing
Stage 3: Grouping & Labeling
Stage 4: Application
2. Feedback Model
Professor Professor Professor Professor & Students who edit to clarify
22
Students have high level of flexibility and creatively to develop rubric task expectations
Rubric ConstructionModel
Stage 1:Reflecting
Stage 2: Listing
Stage 3: Grouping & Labeling
Stage 4: Application
3. Pass-the-Hat Model
Professor Professor & Students
Professor & studentswho group student contributions
Professor & Students who create final rubric
23
Provides more control to students who create both dimensions and some of the descriptions
Rubric ConstructionModel
Stage 1:Reflecting
Stage 2: Listing
Stage 3: Grouping & Labeling
Stage 4: Application
4. Post-It Model
Professor Students Professor & students who facilitate grouping
Professor & Students who create final rubric
24
Faculty role limited to setting the assignment, explaining what the finished rubric will resemble, and facilitating the creation by students
Rubric ConstructionModel
Stage 1:Reflecting
Stage 2: Listing
Stage 3: Grouping & Labeling
Stage 4: Application
5.4X4 Model
Professor & students
Students Professor & studentswho facilitate grouping
Students
25
Criteria 1Unacceptable
2Acceptable
3Good/Solid
4Exemplary
Clarity of criteriaCriteria being assessed are unclear, inappropriate and/or have significant overlap
Criteria being assessed can be identified, but are not clearly differentiated or are inappropriate
Criteria being assessed are clear, appropriate and distinct
Each criteria is distinct, clearly denoted and fully appropriate for the assignment(s)/course
Distinction between Levels
Little/no distinction can be made between levels of achievement
Some distinction between levels is evident, but remain unclear
Distinction between levels is apparent
Each level is distinct and progresses in a clear and logical order
Reliability of ScoringCross-scoring among faculty and/or students often results in significant differences
Cross-scoring by faculty and/or students occasionally produces inconsistent results
There is general agreement between different scorers when using the rubric (e.g. differs by less than 5-10% or less than ½ level)
Cross-scoring of assignments using rubric results in consistent agreement among scorers
Clarity of Expectations/
Guidance to Learners
Rubric is not shared with learners
Rubric is shared and provides some idea of the assignment/ expectations
Rubric is used to explicitly introduce an assignment and guide learners
Rubric serves as primary reference point for discussion and guidance for course/assignment(s) as well as evaluation of assignment(s)
Support of Metacognition
(Awareness of Learning)
Learners do not see/know of the rubric
Rubric is shared but no further reference is made to it in the course/ assignment(s)
Rubric is shared and identified as a tool for helping learners to understand what they are learning through the assignment/ in the course
Rubric is regularly referenced and used to help learners identify the skills and knowledge they are developing throughout the course/ assignment(s)
Engagement of Learners in Rubric Development/ Use *
Learners are not engaged in either development or use of the rubrics
Learners offered the rubric and may choose to use it for self assessment
Learners discuss and offer feedback/input into the design of the rubric, and are responsible for use of rubrics in peer and/or self-evaluation
Faculty and learners are jointly responsible for design of rubrics and learners use them in peer and/or self-evaluation
A Rubric for RubricsA Tool for Assessing the Quality and Use of Rubrics in Education
*Considered optional by some educators and a critical component by othersScoring chart: 0 - 10 = needs improvement 11 - 15 = workable 16 – 20 = solid/good 21 - 24= exemplary
© 2002 BBMullinix ([email protected], [email protected])web url: http://www.tltgroup.org/resources/Rubrics.htm
Timely feedback Prepare students to use detailed feedback Encourages critical thinking Facilitate communication Helps faculty refine teaching methods Consistently levels the “playing field”
27
Carnegie Mellon. Grading and Performance Rubrics.◦ http://www/cmu.edu/teaching/designteach/teac
h/rubrics.html Fink, DL. Creating Significant Learning Experiences.
2003; San Francisco. Jossey-Bass Publishers. Schreyer Institute for Innovation in Learning
http://www.inov8.psu.edu/toolbox/RubricBasics.pdf
Stevens DD, Levi AJ. Introduction to Rubrics. 2005. Sterling, VA. Stylus Publishing Co.
2002 BBMullinix ([email protected])◦ http://www.tltgroup.org/resources/Rubrics.htm
28