A Comprehensive Approach to Designing Online Courses

Creating Assessments Christina Sax

ACADEMIC ■ IMPRESSIONS

• Appreciate the process, importance, and utility of assessment

• Understand the differences between assessment, evaluation, testing, grading

• Explore the various factors that influence the creation of assessments

• Review the basic principles of creating assessments• Explore various aspects of assessment in the online

environment

Session Goals

• What is assessment?

• Why is it important?

Understanding Assessment

Activity Sheet 1

An ongoing systematic process that:• Sets clear expectations of student learning in

the form of objectives• Provides sufficient opportunities for students

to achieve these expectations• Gathers evidence of student learning• Uses this information to improve teaching

and learning

Assessment (verb) – A Dynamic Process

Handout 1

An instrument used to:• Identify what students have learned

• Measure the match between the learning objectives and students’ attainment of those objectives

Assessment (noun) – Strategies & Tools

Handout 1

• Provides feedback to students– identify strengths/weaknesses– adjust learning practices & strategies

• Documents individual student achievement & progress

• Impacts decision making of prospective students

Importance to Students

• Provides feedback for improvement of teaching, support systems, curriculum

• Ensures pre-reqs in course sequencing• Documents success of curriculum• Supports strategic planning/decision making

(funding, resource allocation, etc)• Supports reaccreditation• Addresses external call for accountability

Importance to Faculty and Institutions

Learning Objective (what students should be able to do)

?? ?? Assessment

Learning Outcome (what students can actually do)

Measuring the Gap

• Assessment

• Evaluation

• Testing

• Grading

What’s the Difference?

Activity Sheet 2

1. Evaluation = Judgment• Interpreting assessment results & using them to

make a decision• Investigating the quality or worth of an entity to

make a decision

2. Evaluation = Assessment• Determining the match between objectives and

outcomes

Assessment vs Evaluation

Testing is …

• a systematic measurement of knowledge, skills, abilities

• a part of the assessment process

Assessment vs Testing

• Part of the assessment process

• Grades influenced by:– student learning, attendance, participation, timely

submission of work

• A grade may indicate overall level of understanding, but does not indicate of which specific knowledge, skills, abilities

Assessment vs Grading

1. Focus on learning objectives2. Consider best practices3. Define the purpose4. Select assessment strategies5. Consider the end results6. Link to knowledge/cognition taxonomies7. Select & design assessment tool8. Select & design grading criteria/tools

Steps to Creating Assessments

Step 1 – Learning Objectives

• Align assessments, objectives, questions

• Seek useful information

• Conduct on an ongoing basis

• Use a variety of methods

• Communicate purpose, expectations, grading criteria, feedback loops

Step 2 - Best Practices in Assessment

• Determine pre-req knowledge• Provide learning opportunity• Provide feedback to students• Student self-assessment• Document progress• Document outcomes• Feedback to faculty• Mid-course correction

Step 3 - Define the Purpose

• Formative vs summative• Direct vs indirect evidence• Learning processes vs outcomes• Objective vs subjective• Quantitative vs qualitative• Performance vs traditional• Local vs commercial/published• Embedded vs add-on

Step 4 - Assessment Strategies

• Formative– Occurs while student learning is taking place– Purpose is to improve teaching & learning– Students receive feedback beyond a grade on

strengths & weaknesses

• Summative– Occurs at the end of a course (or program)– Purpose to document student learning– Students may or may not receive feedback other

than a grade

Formative & Summative Assessment

• Objective– Usually one correct answer– Can be lower order and higher order processing– Quickly provide information on broad range or targeted content areas– No professional judgment needed to score– Do not require lots of time to grade– Results easily summarized by a single number

• Subjective– May be multiple possible answers– Assess many important skills at once– Assess skills d irectly– Are more active & promote deep, lasting learning– Students can provide lots of information on broad range of objectives at once– Requires professional judgment to score– Allows for scoring nuances– Require lots of time to score– Summary of results requires narrative

Objective & Subjective – Pros & Cons

• Standards-based – Are students meeting your standards?

• Benchmarking– How do student compare to peers?

• Best Practice– How do students compare to best of their peers?

• Value-Added– Are students improving?

• Longitudinal– Is program improving?

• Capability– Are students doing as well as they can?

Step 5 - Consider the End Results

• Consider level of knowledge and processing

– Lower order vs higher order

• Align objectives (verbs) and assessment(Developing Objectives session – Handout 3)

Step 6 - Link to Knowledge & Cognition Taxonomies

• Objective: Create a sterile environment.• Are these assessments aligned? Why/why not?

1. Write a definition of the phrase “sterile environment”.

2. Arrange these ten steps in the proper order to ensure a sterile environment is achieved.

3. Create an instruction manual titled “Sterile Environment Procedures”.

4. Write an essay describing the importance of sterile conditions in surgical settings.

5. Write an essay describing how you would create a sterile environment under suboptimal conditions.

Align Objectives, Verbs, Assessments

The toolbox includes WAY MORE

than exam questions !!!!

Step 7 - Assessment Tools

Handout 2

• Examples– List of criteria & points– Rubrics

• Uses– States clear expectations; guide for students– Guide to faculty for fair and accurate scoring– Vehicle for feedback to students

Step 8 - Grading Criteria & Tools

• Use to plan, design, and create assessments

• Based in link to specific learning objectives

• State clear and thorough directions and expectations for students

• State grading standards, criteria, rubrics

Assessment Mapping Grid

Handout 3

• Work in teams

• Use Activity Sheet 4

• Review the NURS246 course description & learning objective

• Use the Assessment Mapping Grid to plan, select, and design an online assessment

• Group reports (designate your spokesperson)

Activity – Create an Online Assessment

• Describe the contents of your Assessment Mapping Grid.

• What challenges did you encounter?

• How can you overcome these challenges?

Activity – Group Reports

• Well suited to the online environment

• Not well suited to the online environment

Characteristics of Online Assessments …

Activity Sheet 3B

• Efficient management, collection, transfer of assessment information

• Possibility of reduced grading time• Increased opportunities for students:

– Self-assessment, ongoing & rapid feedback, knowledge of overall status through online gradebook

• Course management system tools better facilitate some assessment practices

Online Assessment – Benefits of Technology

• Database to track, monitor, document• Easy access to development & deployment of

multiple assessment tools with automatic & instant feedback

• Multiple communication tools to – Document & track progress, formally & informally– Provide/receive continuous feedback from faculty &

student peers

• Content tools to develop projects & problem solving tasks, independently & in teams

Course Management System Tools

• Discussions are important form of ongoing informal & formal assessment

• Greater opportunity for full involvement online vs f2f (# students, depth of discussion)

• Easy to track, document, grade student discussion online vs f2f

• Easy to model/provide examples of “quality” responses and dialogue

• Easy to conduct multiple conversations at once

Discussion Boards

• Common online text• Reconstructive writing prompt sends reader back to text• Constructive writing prompt promotes thinking beyond• Inquiry prompt• Directions to interact with fellow students• Expectations for students’ work• Directions for mechanical steps to accomplish tasks• Personal/human feel

Discussion Board Model(Hofmeister & Thomas, 2005)

• Course management system provides tools to structure & facilitate group work online vs f2f

• Allows faculty to monitor progress of group work, and individual contributions to group

• Allows for three forms of feedback – faculty, self, peer

Group/Team Work

Automated grading of objective question typesallows for:

– Ongoing formal assessment– Ongoing informal self-assessment– “Best attempt” assessment– Varied assessment (question types)– Rapid feedback– Adaptation to individual student situations, starting

and progress levels– Reduced grading time

Quizzing/Testing Tools - Objective

• Technology allows for:– Reduced grading time & faster feedback

• Cut & paste common comments, rather than handwritten for each student

– Multiple forms of feedback • Extensive typed text• Hand mark-ups• Audio clips

Quizzing/Testing Tools - Subjective

• What do you want students learn?• What do you want to know about the learning?• Why do you want to know this?• How do you determine this?• How will you use this information?

Assessment requires time, reflection, and planning.

Assessment Wrap-Up