Tests and Quizzes Tool(aka SAMigo)

Marc Brierley, Stanford University T&Q Lead Designer http://bugs.sakaiproject.org/confluence/display/SAM

Agenda

• Feature Overview– Focus on pedagogical flexibility using

templates

• Short Demo

• Futures

• Q&A

What are T&Q’s features:

• Assessment Authoring• Assessment Publishing• Assessment-Taking Management• Assessment Grading• Question Pools Management• Assessment-Type (formerly Template)

Management

Why “Assessments”?

• More than just test and quizzes

• For Example:– Problem Sets– Formative Assessments– Surveys

• The name “Tests and Quizzes” is a compromise

Organize & Publish Assessments

• Core (pending) Assessments for a course– Starting page for assessment authoring– Can be ex/imported in QTI XML format– When ready, published via Settings

• Published Assessments for student assessment-taking– When students respond, scores link appears– Can change release, due, retract dates

Question Types• Multiple Choice (single and multiple correct)• Multiple Choice Survey• True/False• Matching• Essay/Short Answer• Fill in the Blank• File Upload• Audio Recording** New for Sakai 2.2 release

Rich-Text Everywhere

• WYSIWYG editor

Fonts, style, layout, images, HTML, links

• In section headers, question texts, selections for responses, and feedback

Pedagogical Flexibility

• Online test

• Self-study questions

• Homework, problem-sets

• Paper, code, or project submission

• Language drills

• Quick knowledge probes

• Surveys

• Others???

Assessment Types

• Create assessment “types” for different disciplines, different uses

• Hide complexity by selecting what which settings are visible to the instructor.

• Pre-set defaults by making setting choices for the type (e.g., Self-study always has immediate feedback)

Template Workflow

Online Test

• Unique tests for each student– Randomized order of questions– Randomized draw from question pools

• Timed test-taking during access window– Auto-submit at end of timed period

• One submission only per student• Higher Security

– IP Addresses restricted– Secondary Password (proctored)

• No late submissions accepted• Scores transferred to gradebook

Self-Study Questions

• Immediate feedback• Random access to Questions

– Table of contents– Marked for review list

• No record of score in gradebook• No due date; always available to student• Unlimited submissions allowed

Homework/Problem Sets

• Published with multiple release and due dates for different sections

• Multiple methods for handling late submissions• Students can save work during assignment

period.• Auto-grading with methods for diagnosing

learning problems • Quick-reviews of student aggregate performance

by instructors viewing histograms, statistics.• Pools for organizing questions for reuse

Essay and Project Submission

• Use file upload question type for submitting any type of document

• Grader can download assignment; upload marked-up file to return to student

• Grades are recorded in gradebook

Question Pools

• Organize questions into multiple subpools

• Author Questions from within Pool Mgr.

• Randomly draw from pools for an assessment

Demo

T&Q 2.2 Highlights

• Assessment Types• Audio Question Type

T&Q post-2.2 Future

• Attach files/graphics to questions• Export assessment results • Release assessments to specific

classes and sections• Department/school-level assessment

types• Question Pool sharing by group,

import/export• More usability testing

Questions?

Contacts

Marc Brierley, Stanford University T&Q Lead Designer marc.brierley@stanford.edu

Lydia Li, Stanford University T&Q Project Manager lydial@stanford.edu

Homepage:http://bugs.sakaiproject.org/confluence/display/SAM

Where did the design originate?

• Formative Assessment (Homework)– Stanford Assignment Tool for Human Biology– CourseWork CMS with Language Tools

(Andrew W. Mellon Foundation funded)

– OKI Tools Workshop to Specify Functionality(Andrew W. Mellon Foundation funded)

– OKI Formative Assessment Tool(Andrew W. Mellon Foundation funded)

• Summative Assessment (High-Stakes tests)– OnCourse Quiz Tool

Where did the design originate?

• OKI Tools Workshop– Many schools met to define needs for a

CMS– One group specified the needs for an

Assessment Tool– Several hundred functional requirements

identified

Where did the design originate?

• Assignment and Assessment Manager – Supports teaching and learning in

large lecture science courses and in language instruction

– This OKI Tool was developed at Stanford– Andrew W. Mellon Foundation funded

Where did the design originate?

• AAM design combined with OnCourse Quiz tool design “Navigo” (2003)

• Stanford and Indiana join Sakai project, Stanford rewrites tool to Sakai Style Guide TPP, tool renamed SAMigo (2004-present)