Learner Assessment in an Online EnvironmentPatricia McGee, Alisa Cooper, Jennifer Freeman

1. Developed my own assessments

2. Used publisher-created assessments

3. Worked with others to create assessments

4. Used assessments developed by someone other than publisher

Which best describes your experience with online assessments?

Overview of Workshop

Classroom vs. Online Assessment Best Fit Assessments for Online Courses Designing with the End in Mind Giving Assessment Directions &Feedback Rubrics and Checklists Involving the Learner

Asse

ssm

ent T

oolk

it

P. McGee

Classroom vs. Online Assessment

Patricia McGee

Instructional Functions of Assessment

One to many Just in time

assessment Immediacy Focus on methods

One to one Just in need

assessment Ambiguity Focus on strategies

Differences? Time Security Integrity Resources Learner autonomy Lack of just in need

responses Getting, keeping, and

directing attention Clarity of instructions

and expectations Other?

Challenges?

Cheating Randomized objective

quizzes/tests Timed and authenticated Multiple ways to demonstrate

understanding Choices Duplication Repetition Revise and resubmit

Small group assessments for accountability

Communication Detailed expectations, e.g.,

rubrics

Respond within a pre-designated timeframe

Peer critique review before final submission

Benchmarks, points of reflection, accomplishment & completion

FERPA?

Activity

Brainstorm for assessment challenges.

Discussion possible solutions

Be prepared to share a particularly difficult challenge.

Asse

ssm

ent T

oolk

it

P. McGee

Progressive Assessment & BenchmarksPatricia McGee

Learners should know….

That an assessment is coming A word about helping students plan

What happens after the assessment How to prepare for an assessment What participating in the assessment will look

like Practice

Classroom (online) Assessment Techniques

Informal Ungraded Low threat and no risk Inform instructor about student learning Inform learner about learning progress

From Classroom Assessment Techniques

CATS Examples: Prior KnowledgeCATBackground Knowledge Probe: short, simple questionnaires prepared by instructors for use at the beginning of a course or at the start of new units or topics; can serve as a pretest

Online ApplicationSurvey or Poll Survey Monkey™ Poll Everywhere™ PollDaddy™

CATS Examples: Prior KnowledgeCATEmpty Outlines: in a limited amount of time students complete an empty or partially completed outline of an in-class presentation or homework assignment

Online ApplicationShared Notetaking/Concept MappingBubblus™ Mind42™ Mindomo™ Wordle™

5 Suggestions for Online CATs

1. Customize to specific needs and learning environment

2. Be consistent with the instructional philosophy

3. Test out a CAT and assess effectiveness

4. Allow extra time to carry out and respond to the assessment

5. Let students know what you learn from their feedback and how you and they can use that information to improve learning

ACTIVITY Review the CATs and pick

one Identify the goal for your

CAT Explain why this CAT is

helpful/necessary in this particular area of the course

How and when will students receive feedback on the CAT?

Progressive Assessment

Online Progressive Assessment

Interaction Assignments Activities Participation

Performance vs Objective Assessments

Objective Classroom

Assessment Techniques

Multiple choice tests Embedded Test

Questions Pre and Post Tests Discussion Threads

Performance Activity-Based

Cases Projects Debates/

negotiations Production

Field experience Products Presentations Research paper

Example: Periodic Assessments

24

Example: benchmarks

25

Example: progressive benchmarks

26

Example: Assignment

BENCHMAR

K?

Activity: Progressive Assessments

What are most comprehensive benchmarks for a course?

What are challenges? Issues?

Design with the end in mindAlisa Cooper

Provide multiple and alternative modes of assessments

Learning and AssessmentCycle

Bloom’s Taxonomy

Backwards Design

Focus on learner

Focus on measure of learning

Assessment & Bloom’sProcesses Tools Attributes

Remember Recognizing, recalling Visual/Text/Audio stimuli, selecting, feedback

Understand Interpreting, classifying, comparing, summarizing, explaining

Sorting, tagging, labeling, entering, selecting

Apply Executing, implementing Manipulating, entering, feedback

Analyze Differentiating, organizing, attributing

Selecting, grouping, altering, tagging, labeling

Evaluate Checking, critiquing Commenting, entering, responding

Create Generating, planning, producing

Adding, generating, combining, publishing

ACTIVITY Using handout,

determine at what level of Bloom’s you are assessing

Explore tools that might allow you to assess at higher levels of Bloom’s see http://elearningtools.wetpaint.com

Giving Assessment Directions & FeedbackJenni Freeman

Assessment Instructions

This seems like a fundamental concept, but many assessments lack adequate instructions,

leading students to answer questions incorrectly or without the desired depth.

Two levels: How to use the tools, logistics Student expectations

Assessment Instructions: Questions to Answer

Objective Tests Using the tool? How many times available? When available? Results available? Other?

Assessment Instructions: Questions to Answer

Subjective Assessments How to submit? When due? How will it be graded? What does a successful paper look

like? Results available? Other?

Assessment Instructions: Best Practices• Separate general instructions (How to use the tools, logistics) from assessment-specific instructions (due dates, rubrics)

Centrally located pop-up Syllabus Designated discussion forum Provide grading rubrics Provide models if appropriate (an “A”

paper and an “F” paper)

Assessment Feedback

The sooner the better Instant results (objective tests) Auto-reply upon receipt (subjective assessments)

Details Feedback for all distracters, not just the wrong answers

(objective - students who guess correctly will also receive benefit)

Compose boilerplate language and phrases for common comments to students (subjective)

Assessment Feedback

The sooner the better Instant results (objective tests) Auto-reply upon receipt (subjective assessments)

Details Feedback for all distracters, not just the wrong answers

(objective - students who guess correctly will also receive benefit)

Compose boilerplate language and phrases for common comments to students (subjective)

ACTIVITY

Write instructions for your sample assessment keeping all of the above in mind. Share with your table.

What sort of feedback will your students need after completing the assessment?

Frequency of Feedback

Pedagogical Timing After practice

At completion/achievement of objective

Calendar Schedule Indirect – weekly

Direct - bi-monthly

For exampleChat – content learning Answer questions

correctly

Ask questions that Helped others learn Extended discussion

Make contributions to discussion that extended materials

Chat - performance Participate throughout

discussion

Do not interrupt

Respond when called upon

When appropriate, come to chat prepared.

Outcome + Criteria = Assessment1. How are you communicating that to learners?

2. How will you document the learning ?

3. Do you expect learner to learner a new tool for assessment ?

4. Have you provided for practice before assessment?

Using Rubrics Allows assessment to be more objective and consistent

Focuses instruction to clarify criteria in specific terms

Clearly shows the student how their work will be evaluated and what is expected

Rubric Tips

Ensure that the assessment criteria can be directly correlated to the course objectives.

Break the criteria down as specifically as possible and clearly define exemplary, satisfactory, and unsatisfactory efforts for each.

Be sure that students have access to the rubric before they begin the assignment.

Activity Review samples of rubrics. Identify the characteristics that need to be evaluated in

your assessment. Describe the best work you could expect using these

characteristics: top category Describe the worst acceptable product using these

characteristics: lowest category Describe intermediate work using these characteristics Build a rubric online using Rubistar

http://rubistar.4teachers.org Share your rubric with your table.

Involving the LearningAlisa Cooper

Peer Assessment

Contributes to…. Ownership and deeper

learning

Motivation

Independent and creative thinking

Self awareness of learning as compared to that of others

Strategies Participation in peer

assessment scales, processes

Rating

Peer Critique/Review

Mentoring/tutoring/coaching

Rubrics: for drafts, in conjunction with instructor assessment

Self-assessment

Incorporate meta-cognitive assessments

Provide a strategy for self-assessment and progress

Track progress Compare work to that

of others Grade own

performance

Self-Assessment

Techno CATs

Before/after unit

Reflection

Feedback on design

Feedback on technology

Increases accountability

Engages students

Starts discussions

Early alert

Practice

Progressive Benchmarks for the Learner

Examples

1. What was the one most useful thing you learned in this assignment, unit or module?

2. What suggestions would you give other students on ways to get the most out of this assignment, unit or module?

3. In what area did you learn or understand the most? Least? 4. List three ways you think you have developed or grown as a result of

this assignment, unit or module?5. What did you learn about writing, research, (or any other skill) from

this assignment, unit or module?6. What problems did you encounter in this assignment, project, unit, or

tool that was used?7. What unit/module of this course was your best work and why?

Team Assessment

Roles Teams

Final outcome against pre-determined criteria

Completion Performance against

other teams

Individual

Contribution

Objective assessments

Accountability Require pre-group work Require group members to

express individual opinions and monitor via another member

Include peer evaluation in grading Readiness Assurance Process

Test over readings Group: Test, discuss, reach

consensus and retest Provide information for peer

feedback process

Team Teaching Tips

Outline learning goals Teach team skills Clear and detailed

instructions Rubric Stages of team

development Forming - polite but

untrusting Storming - testing others Norming - valuing other

types Performing - flexibility

from trust 59

60

Team Contracts

Purpose, goals, and missions

Expectations

Roles

Conflict resolution strategies

Meetings

Communication

Decision-making policy

Agendas

Record-keeping

Poll/Survey Options

Functions Engage – collect data –

generate conversation

Informal assessment of understanding/comprehension

Formal assessment of critical thinking

Strategy Ideas Random questions for

points

Drill & practice activities with participation points (online quizzes)

Student constructed – collaborative/cooperative

Matrix survey

http://elearningtools.wetpaint.com/page/Polling%2C++Surveying

Take Aways

What NOT to do

Require a different or separate assessment for each use of tool

Assume the learner knows what to do Assume the fun factor over rides the learning

requirements

Evaluate your assessment practices regularly

Are their persistent problems? Is there an area of the course that gets “stuck” and

remediation or review is needed? Is a lot of time spent on answering questions or

clarifying?