Month, XX YEAR (Arial 10)
Evaluating Student Learning – Ted Scholz
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
2
Evaluating Student Learning: Evidence Based Teaching
Formative Classroom Assessment
• Ongoing• Intended to improve
performance (yours and your student’s) throughout the term (developmental)
• It is used internally to make adjustments• Classroom Assessment T
echniques• Low/no stakes writing
Summative Classroom Assessment
• Occurs at the end of the course
• Intended to determine whether or not overall goals have been achieved
• Used for internal or external accountabilty purposes
• Grades
(Middle States, 2008, pg 27)
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
3
Huba and Freed (2000) remind us, “When we assess our students’ learning we force questions, ‘What have our students learned and how well have they learned
it? How successful have we been at what we are trying to accomplish?’” (p 8).
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
4
Question?
What is a more valuable method of assessment?
• A student completes a calculus problem correctly and shows her work.
• A student describes her own calculus abilities as excellent.
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
5
Indirect and Direct Methods of Assessment (Middle States, 2008, pg 29)
Direct Measures• Coursework and homework
• Examinations and quizzes
• Standardized tests
• Term papers and reports
• Internship performance
• Research projects
• Case study analysis
• Rubric scores
• Class Discussion participation
Indirect Measures• Course Evaluations
• Percent of class time spent on active learning
• Number of hours spent on service learning
• Test blueprints (outlines of the concepts and skills covered on tests)
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
Direct Forms: Strengths and Limitations
Strengths
• They provide evidence of what students learned from the educational experience.
• To what degree they learned.
• What students did not learn.
Limitations
• They provide no evidence as to why the student has learned.
• They provide no evidence as to why the student has not learned.
(Walvoord, 2004, pg 13)
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
Indirect Assessment at TCS
Formative, Early-Term Evaluation
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
Key Questions to Consider for Instruments(Middle States, 2008, pg 39)
• Is the evidence linked to important learning outcomes (stated in syllabus).
• Is a standardized instrument appropriate?
• How comprehensive do I want to make it?– Should I assess by multiple means?
• Does everyone interpret the responses the same way?– If it walks like a duck….
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
9
Why Use Rubrics?
• Provide timely feedback
• Prepare students to use detailed feedback
• Encourage critical thinking
• Facilitate collaboration and communication with others
– Consistency!• Help us refine teaching skills
• Level the playing field
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
10
Level-Dimension Rubric(Stevens & Levi, 2005, p 6 )
Scale Level 1 Scale Level 2 Scale Level 3
Dimension 1
Dimension 2
Dimension 3
Task Description:
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
11
Scoring Guide Rubrics(Stevens & Levi, 2005, p 118 )
Criteria Comments
Dimension 1
Dimension 2
Dimension 3
Task Description:
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
12
Two Types of Rubrics
• Three to Five Level Rubrics– Circle text– Check boxes
• Most time consuming to make
• Fastest to use
• Scoring Guide Rubrics– Designed to give narrative, specific feedback
• Easiest to create
• Most time consuming to grade (particularly with weaker students)
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
Creating Rubrics
• Stage 1: Reflecting– Pgs 12-13 in you packet
• Stage 2: Listing and Matching– Pg 14 in your packet
• Stage 3: Grouping– Pg 14
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
Creating Rubrics (Continued)
• Stage 4: Application– Begin with your highest level first– Scoring guide? You’re finished!– Three to five level:
• Work from the outside in
• Middle will be most difficult
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
Scoring
• Consistent Performance Anchors• Detailed, Formative Feedback
– Three to five level rubrics with boxes
– Three to five level rubrics with circled text • Individualized, Flexible Feedback
– Scoring guide rubrics
• Summative Feedback and Grades– Quantify dimensions?
• Danger of fighting over points
– Before rubrics, it was narrative and a grade– Holistic judgments
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
“The best way to explain a rubric is to show one”
(Walvoord, 2004, p 81)
The Chicago School of Professional Psychology ▪ proprietary and confidential information ▪ all rights reserved
References
Bain, Ken (2004) What the Best College Teachers Do. Cambridge: Harvard UP
I-Rubric (2010) R Campus: open tools for open minds. Accessed on January 26th, 2010. http://www.rcampus.com/indexrubric.cfm
Huba, M.E., & Freed, J.E. (2000). Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Boston: Allyn & Bacon
Middle States Commission on Higher Education. (2007). Student Learning Assessment: Options and Resources. Philadelphia: Middle States Commission. www.msche.org
Stevens, D.D. & Levi, A.J. (2005) Introduction to Rubrics. Sterling VA: Stylus. http://styluspub.com/resources/introductiontorubrics.aspx
Walvoord. B.E. (2004). Assessment Clear and Simple. San Francisco: Jossey-Bass.