Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Using Learning Assessment Techniques
Throughout Your Course
A workshop for
Santa Monica College FSI 2017
August 9, 2017 - AM
Kevin Kelly, EdD This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
About me Since 1999 • Teaching at San Francisco State University
• Current class: UG, Gen Ed course "How 2 Lrn w ur Mobile Device"
• Faculty development/Leadership development • San Francisco State (several hats) • Wiley Learning Institute • Association of College and University Educators (ACUE) • Higher education consulting
2 FSI 2017 Workshop - Assessment
About this workshop What we'll do today: • assess and adjust learning outcomes from one or more
courses • use the Learning Goals Inventory to identify pedagogical
areas of emphasis • discuss structuring activities that promote both
assessment for learning and assessment of learning • work in small groups to explore different categories of
assessment techniques that support common goals • practice analyzing the results from assessments • commit to trying at least one assessment technique in Fall
2017
FSI 2017 Workshop - Assessment 3
FSI 2017 Workshop - Assessment 4
NOTE: Assessment is a big topic! • We WILL NOT attempt to drink
straight from a fire hydrant. • We WILL pick a few practical
strategies to use in your course over time.
Image: "fire hydrant" CC BY Renee
About you - setting personal goals
What is one goal you want to achieve during this workshop about assessment?
For the next 2 to 3 minutes: 1. Form small groups & assign a "tech scribe" *
• * someone with a laptop or mobile device 2. Visit Today's Meet:
https://todaysmeet.com/FSI2017-assessment 3. Enter each person's goal as a new entry
5 FSI 2017 Workshop - Assessment
About you - setting personal goals
What is one goal you want to achieve during this workshop about assessment?
Common goals:
6 FSI 2017 Workshop - Assessment
About assessment
What do you associate with assessment in your class(es)?
For the next 5 minutes: 1. 2 min: On your own, list 5 to 7 things that you
associate with assessment in your class(es). 2. 3 min: Turn to a neighbor and compare your
lists. Identify any items you have in common.
8 FSI 2017 Workshop - Assessment
About assessment
What do you associate with assessment in your class(es)?
Common: option to work in pairs, rubrics, check for understanding, disconnect btw what we think they know and test results, stress, scaffold group activities (guided questions), sufficient time, goals for both students/teachers, retention (in context of the test), essays, pre-assessment, student questions,
9 FSI 2017 Workshop - Assessment
Pulling back the curtain • Learning Goal Listing (LAT 47)
• Goal: Help learners become aware of their own goals.
• Timing: Use at the start of a course or unit. • Tips: Help learners connect their goals to
course goals and/or student learning outcomes.
• Focused Listing Activity (CAT 2) • Goal: Determine what learners know or recall
about a topic. • Timing: Use at the start of a class meeting. • Tips: Ask small groups to share common items.
Get a show of hands for each item. Share missing items from your own list.
10
What is assessment? • Barkley & Major (2016, p. 24) define "assessment" as
"appraisal of knowledge, skills, attitudes and beliefs that students have acquired, most often as a result of learning in their courses."
• Assessment is different from grading. Among other things, assessment focuses on a group of students, while grading focuses on an individual student.
FSI 2017 Workshop - Assessment 11
What is assessment? • For the purposes of this workshop, we'll look at
assessment from a number of angles • What can we actually assess? • Why use a specific assessment strategy? • When should we use different assessment strategies? • Who should facilitate the assessment? • How do we implement those assessment strategies? • Can we implement the assessment strategies with technology? • What happens after we conduct an assessment?
• We'll focus on ways to engage learners with practical assessment activities you can use right in the classroom.
FSI 2017 Workshop - Assessment 12
Assessment references (and resources) Learning Assessment Techniques
by Barkley & Major (2016) Classroom Assessment Techniques
by Angelo & Cross (1993)
13 FSI 2017 Workshop - Assessment
Assessment references • Since I'll refer to both resources throughout this workshop,
let's see what they are: • Learning Assessment Techniques (LATs) are comprised of three
parts, reflecting and incorporating effective teaching elements—identify goals, facilitate activities, and assess outcomes.
• Classroom Assessment Techniques (CATs) "are generally simple, non-graded, anonymous, in-class activities designed to give you and your students useful feedback on the teaching-learning process as it is happening" (Vanderbilt CFT, n.d.).
• Both books outline a process AND provide 50 practical techniques!
FSI 2017 Workshop - Assessment 14
What can we assess? Learning Assessment Techniques
Cognitive Domains • Foundational Knowledge • Application • Integration Humanistic Domains • Human Dimension • Caring • Learning How to Learn
Classroom Assessment Techniques
Knowledge & Skills • Prior Knowledge, Recall • Analysis, Critical Thinking • Synthesis, Creative Thinking • Problem Solving • Application, Performance Learner Attitudes, Values, Reactions • Awareness of Their Attitudes & Values • Self-Awareness as Learners • Study Skills, Strategies & Behaviors • Reactions to Teachers & Teaching • Reactions to Class Elements
FSI 2017 Workshop - Assessment 15
Why should we assess? • Determine for ourselves how well students are learning
• So we can make any necessary changes
• Give learners feedback on their progress • So they can make necessary changes before high-stakes exams
• Improve our profession through the Scholarship of Teaching and Learning • Conversations with colleagues, publish an article
• Provide information to institutional and external stakeholders on how well students are learning • E.g., Program review, Accreditation
(Barkley & Major, 2016, pp. 26-28)
FSI 2017 Workshop - Assessment 16
STRETCH BREAK (5 MIN)
Image CC BY-SA Quinn Dombrowski
For the next activity, we'll create different groupings…
Time to move! (discipline clusters)
• Art, Arts, Design • Humanities • Languages
• Social Sciences • Comm & Media • Psychology
• Sciences • CTE programs
• Technology
• Business • Professional Training
• Workforce Dev
FSI 2017 Workshop - Assessment 18
WHERE TO START? SETTING THE STAGE FOR ASSESSMENT • (Re)write or add learning outcomes for 1 or more
modules • Plan assessments for each outcomes (along with
activities and materials)
19 FSI 2017 Workshop - Assessment
“The key to success is to appreciate how people learn, understand the
thought process that goes into instructional design, what works well,
and a range of different ways of achieving goals.”
- Tim Buff
20 FSI 2017 Workshop - Assessment
Activity “The key to success is to appreciate how people learn, understand the thought process that goes into instructional design, what works well, and a
range of different ways of achieving goals.” - Tim Buff
• Paraphrase: Rephrase the passage in your own words. • Interpret: Explain the meaning of this passage. • Comment: Express your opinion of this passage. • Cite: Write out how you would cite the quote using APA,
MLA or other.
21 FSI 2017 Workshop - Assessment
Activity “The key to success is to appreciate how people learn, understand the thought process that goes into instructional design, what works well, and a
range of different ways of achieving goals.” - Tim Buff
From a volunteer • Paraphrase: • Interpret: • Comment: • Cite:
22 FSI 2017 Workshop - Assessment
Activity “Think about what your learners need to do with that information
after the course is finished and design around that.”
- Matthew Guyan • Summarize this quote in one word. • Explain: In 1-2 sentences, explain why you chose that
word.
23 FSI 2017 Workshop - Assessment
Activity “Think about what your learners need to do with that information
after the course is finished and design around that.”
- Matthew Guyan From a volunteer • Summarize: • Explain:
24 FSI 2017 Workshop - Assessment
Pulling back the curtain • Quotation Commentaries (LAT 13)
• Goal: Help learners work with quoted material from assigned readings.
• Timing: Use to prepare for writing assignments. • Tips: Take time to showcase the value in what
learners are asked to read for your class.
• Word Journal (CAT 14) • Goal: Stimulate synthesis and creative thinking. • Timing: Use when learners should read carefully. • Tips: This strategy works best for reading short text
(essays, poems, short stories, articles, case studies).
25
Where to start? Objectives vs Outcomes
Learning objectives
• Objectives are a target—what we hope students will do
Learning outcomes
• Outcomes are an end result–students showing what they did • Observable, measurable • Examples: Student Learning
Outcomes, Institutional Learning Outcomes
FSI 2017 Workshop - Assessment 26
(Barkley & Major, 2016)
(Re)writing learning objectives/outcomes
• Start with the usual: What should students be able to do by the end of the course?
Example objectives:
• Use metacognitive strategies designed to improve your learning
• Explore different technologies specifically for learning
27 FSI 2017 Workshop - Assessment
(Re)writing learning objectives/outcomes
• Add/revise objectives/outcomes for scope, specificity (granular): • course level • module/unit level • activity level (?)
28 FSI 2017 Workshop - Assessment
(Re)writing learning objectives/outcomes
• Add/revise outcomes for scope, specificity: • Example: Explore different technologies
specifically for learning • Through a quiz, show that you know what a technology
is, what it does, and how it can be used for learning. • In a discussion forum, describe how you used a
technology for specific learning activities. • Given a curated set of links, complete an evaluation
sheet outlining how effective a specific technology is for your own learning.
• Write an essay that reflects on the effectiveness of using a technology for learning in one of your other classes or for work-related training.
29
Course objective
Module outcomes
FSI 2017 Workshop - Assessment
(Re)writing learning objectives/outcomes • Rethink the level
(of thinking, of achievement): • Bloom's taxonomy
(updated)
30 FSI 2017 Workshop - Assessment
(Re)writing learning objectives/outcomes
• Identifying types of objectives/outcomes • Cognitive (knowledge)
• Example verbs: define, explain, predict, compare, evaluate, design
• Psychomotor (skills) • Example verbs: watch, operate, write, perform
• Affective (attitudes or feelings) • Example verbs: select, discuss, justify, integrate, revise
31 FSI 2017 Workshop - Assessment
•Cognitive (knowledge) - Bloom’s Taxonomy (revised) •Remembering: Recall/recognize specific information •Understanding: Lowest level of understanding •Applying: Implement (use) information •Analyzing: Break information into parts and describe the relationship •Evaluating: Make a judgment about materials or methods •Creating: Generate new ideas, products and ways of viewing things
(Re)writing learning objectives/outcomes
See http://itc.utk.edu/~bobannon/ classifications.html#cognitive_domain
Image source: http://www.iftf.org
32 FSI 2017 Workshop - Assessment
•Psychomotor (skills) •Observing: Active mental attending of a physical event •Imitating: Attempt to copy a physical behavior •Practicing: Try a specific physical activity over and over •Adapting: Fine tuning, make minor adjustments in a physical activity
(Re)writing learning objectives/outcomes
See http://ets.tlt.psu.edu/learningdesign/ objectives/psychomotor
Image source: http://www.sharjah.ac.ae
33 FSI 2017 Workshop - Assessment
•Affective (attitudes or feelings) •Receiving: Shows awareness. •Responding: Shows attention and motivation. •Valuing: Demonstrates commitment. •Organizing: Integrates and prioritizes new values. •Characterizing by value: Behavior reflects new values.
(Re)writing learning objectives/outcomes
http://www.uwsp.edu/education/lwilson/ curric/affectiv.htm
Image source: Karin Kirk, SERC http://serc.carleton.edu
34 FSI 2017 Workshop - Assessment
Activity Task (3-5 min): • Review your course objectives or outcomes • (Re)write one course-level objective • Add one or more unit or module-level outcomes
35 FSI 2017 Workshop - Assessment
Activity: Share • 3 to 5 faculty will share before/after versions of their objectives
36 FSI 2017 Workshop - Assessment
Backward course design • Now that you have your objectives,
it's time to determine how students will show what they know.
• Then select activities that will prepare them to succeed with the assessment strategies.
• Last, identify the content that they'll need for all of it.
37
Image: UC Davis
FSI 2017 Workshop - Assessment
Why should we assess? • Determine for ourselves how well students are learning
• So we can make any necessary changes to curriculum
• Give learners feedback on their progress • So they can make necessary changes before high-stakes exams
• Improve our profession through the Scholarship of Teaching and Learning • Conversations with colleagues, publish an article
• Provide information to institutional and external stakeholders on how well students are learning • E.g., Program review, Accreditation
(Barkley & Major, 2016, pp. 26-28)
FSI 2017 Workshop - Assessment 40
When should we assess? • Formative: Checking for student understanding during
the learning process • E.g., low-stakes quizzes or assignments
• Summative: Evaluating what students learned after • E.g., midterm or final exams, final projects or essays
• Confirmative: Checking to see what students remember over time, well after the learning took place • E.g., At the start of a term, use a Background Knowledge Probe
(CAT 1) to see what prerequisite information students remember
FSI 2017 Workshop - Assessment 41
Prerequisite class – Final exam
(Summative) Break between
semesters
First class – Background knowledge
probe (Confirmative)
Quizzes and assignments (Formative)
In-class assessment strategies
(Formative)
Midterm or Final exam
(Summative)
When the cook tastes the soup, that’s formative; when the guests taste the
soup, that’s summative. - R. Stake
FSI 2017 Workshop - Assessment 42
Why and when are we assessing?
Assessment FOR learning
• Formative • Supports/advances
student learning
Assessment OF learning
• Summative • Confirms student
achievement
FSI 2017 Workshop - Assessment 43
Activity • Draw a concept map showing the connections between
the major ideas from this workshop. • Incorporate other concepts you have learned at FSI this week.
OR • Map a sequence of assessment events for your class.
• Show which activities are assessment OF learning and which are assessment FOR learning.
44 FSI 2017 Workshop - Assessment
Activity • Explain your sequence chains or concept maps to a
neighbor (2 min each).
FSI 2017 Workshop - Assessment 45
Pulling back the curtain • Sequence Chains (LAT 22)
• Goal: Help learners apply what they've learned to arrange items in order.
• Timing: Use as either formative or summative assessments.
• Tips: Take time to showcase the connections. Promote the end products as study aids.
• Concept Maps (CAT 16, LAT 23) • Goal: Synthesize, organize and show connections. • Timing: Use to assess prior knowledge or estimate
learning growth. • Tips:.
46
Which assessment technique to use? Learning Assessment Techniques
by Barkley & Major (2016)
• Learning Goals Inventory correlates to Fink's Significant Learning Taxonomy • Helps teachers 1) articulate
goals for student learning, 2) choose appropriate LATs, and 3) foster discussions re: teaching & learning with colleagues
Classroom Assessment Techniques by Angelo & Cross (1993)
• Teaching Goals Inventory correlates to the types of skills, knowledge & attitudes you believe are essential • Helps teachers 1) identify
what they want to achieve in courses, 2) choose appropriate CATs, and 3) foster discussions re: teaching & learning with colleagues
FSI 2017 Workshop - Assessment 48
Activity Today we'll use the Learning Goals Inventory: • Part 1: Select a course that you'll teach in the Fall.
• Feel free to work on the same course for which you revised learning objectives or outcomes.
• Part 2: For that course, rate the importance of each item in the Learning Goals Inventory: • Rating scale: 5 high, 4 Moderately high, 3 Moderately low, 2 low, 1
N/A
• Part 3: Answer the Learning Goal Self-Assessment questions
FSI 2017 Workshop - Assessment 49
Activity Now let's score the Learning Goals Inventory: • Add up the subtotal scores for each domain to determine
which domains have the highest value to you. • Identify the most important cognitive and humanistic
aspects of learning you rate for your students.
FSI 2017 Workshop - Assessment 50
Time to move! (LGI clusters)
• Foundational knowledge (front, left)
• Application (middle, left)
• Integration
(back, left)
• Human dimension (front, right)
• Caring (middle, right)
• Learning how to learn (back, right)
FSI 2017 Workshop - Assessment 51
Join your peers based on the most important domain identified from the LGI
Activity Using the activity worksheet: • First, read the description of the Learning Assessment
Technique for the most important Domain you identified through the Learning Goals Inventory (2-3 minutes)
• Then, begin work on the task designed to prepare you for implementing the technique (5-10 minutes)
• Last, each person presents his/her draft to the group (2 minutes each person)
FSI 2017 Workshop - Assessment 52
Time to move again! (LGI clusters)
• Foundational knowledge (front, left)
• Application (middle, left)
• Integration
(back, left)
• Human dimension (front, right)
• Caring (middle, right)
• Learning how to learn (back, right)
FSI 2017 Workshop - Assessment 53
Join your peers based on your second most important domain (or one that interests you)
Activity Using the activity worksheet: • First, read the description of the Learning Assessment
Technique for the most important Domain you identified through the Learning Goals Inventory (2-3 minutes)
• Then, begin work on the task designed to prepare you for implementing the technique (5-10 minutes)
• Last, each person presents his/her draft to the group (2 minutes each person)
FSI 2017 Workshop - Assessment 54