Comparing Cookies to Courses

Best Practice Session at Detroit LakesBased on Quality Matters Project Resources from the

Maryland Online ConsortiumDeborah Proctor, Ph.D

Deborah.proctor@csu.mnscu.eduE-Curriculum Director for Minnesota Online

Session Objectives

• Develop a rubric by:

– Examining different brands of cookies.

– Determining the criteria for review.

– Setting the standards for the review.

• Reflect on the process of determining quality issues and standards.

• Discuss the implications for the process in online course review.

Quality Matters

How do we …

– identify & recognize it?

• Determine the Criteria of the Reviewers?

• Develop a Rubric on Quality?

Zen and the Art of Motorcycle Maintenance

• ""Even though quality cannot be defined, you know what quality is."

• Robert M. Pirsig - Zen and the Art of Motorcycle Maintenance

Develop a Rubric

1. Determine the criteria: Who can review?– People with some

experience in eating.

2. Examine the “goods.”

– Open the different

bags of chocolate chip cookies.

Developing Standards

3. Establish the weighted criteria of the “goods”. These are your standards.

• What are the criteria?• What weight will you

give each criteria?

Rating the Goods

4. Collecting data– How did you test the

cookies for quality?– How did you record

the data? – What are your cookie

ratings?

Revisiting the Cookie Quality Criteria

• What were the deciding factors for quality in your group? The deal breakers?

• What insights to creating a rating rubric and process can your group share?

• What revisions did you make to your original rating rubric as you compared cookies?

The Reflection

1. Did your group achieve consensus on the ratings? How?2. What were the greatest challenges to arriving at a

consensus ranking?3. How comfortable would you be if you had to defend this

rating before your vice president or before the faculty association?

4. Can your criteria be applied to all such similar items?5. Is this process replicable? Can you use this process for

other quality assurance activities?6. Based on your cookie review what advice would you

give to quality assurance teams?

Cookies Don’t Just Happen!

• Recipe• Ingredients• Processes• Packaging• Marketing• Seal of Approval

How is the process for creating quality cookies similar to quality online courses?

AnalogyCookies to Online Course Review

Issues Cookies Online Courses

How long was the Time on Task?

Who sets the criteria?

Who determines the weighting standards?

Is the process replicable?

Are indicators subjective? Objective?

Are there “deal breakers?”

How much can be evaluated at each sitting?

What are the experiences of the reviewers?

What were the unifying elements? J.D.

Mcmahon, Towson University, MD mcmahon@towson.edu

Quality Matters Project

QM Key Outcomes

1) Replicable pathway for inter-institutional quality assurance (QA) of online courses

2) Faculty-centered, peer review-based, consortium-wide quality

matters process

3) Review tools that incorporate nationally recognized standards of best practice

4) Face to face and online training

5) Expanded resource sharing and increased articulation agreements

Connecting Outcomes to Peer Review

Peer Course Peer Course ReviewReview

FeedbackFeedback

CourseCourse

Course Meets Course Meets Quality ExpectationsQuality Expectations

Course Course RevisionRevision

Instructional Designers

InstitutionsFaculty Course Developers National Standards &

Research Literature

Rubric

Faculty ReviewersTraining

Quality Matters:Quality Matters:

Peer Course Peer Course Review ProcessReview Process

What QM Peer Review is not!

• Not about an individual instructor (it’s about the course)

• Not about faculty evaluation (it’s about course quality)

• Not about “winners” and “losers” (it’s about continuous improvement in a supportive environment)

For Maryland Online Purposes, Quality Was Described as…

• More than average; more than “good enough”

• An attempt to capture what’s expected in an effective online course

• Based on research and widely accepted standards

Higher Standard

100

50

0

For Your Online Purposes, Quality Is? Discuss…

• More than average; more than “good enough”?

• An attempt to capture what’s expected in an effective online course?

• Based on research and widely accepted standards?

100

50

0

Zen and the Art of Motorcycle Maintenance

• "What he meant by Quality was obvious. They obviously knew what it was too, and so they lost interest in listening. Their question now was "All right, we know what Quality is. How do we get it?"" –

• Robert M. Pirsig - Zen and the Art of Motorcycle Maintenance

Inter-Institutional Sharing

• Minnesota Online is a statewide consortium of 32 community/ technical colleges and state universities offering a wide variety of courses & programs available in our system and we don’t share much…yet

Maryland Online (MOL) is a statewide consortium of 10 Maryland community colleges and senior institutions that share courses.

What Are the Challenges?

• Inter-institutional agreement

• Increased articulation agreements

• Peer Review Process

• Adoption of rubric tools and/or standards

• Faculty training and buy-in

• Expanded sharing of resources

Scaffolds & Ladders

• Great news! You do not have to start at the bottom.

• Qualitymatters.org• ITEACH Online Cente

r• Merlot.org

What’s the Catch?

• Using QM Tools & Resources– Let them know you are

using tools– If you revamp or revise

resources or tools…share the materials with them

– Let them know success and challenges of projects related to establishing QM

Quality Matters:QA Process Overview

• Review of online courses by teams of 3 faculty members

• Using Rubric tied to national standards of best practice

• Feedback provided for improvement of course quality

• Instructional design support provided to implement recommendations

• Certification of course when standards met

Goals for Selecting Courses

• Focus on quality• Continuous improvement• Positive environment• Low-risk• Increase number of

“shareable” courses