SAMigo Assessment ToolSAMigo Assessment Tool(aka Tests and Quizzes )(aka Tests and Quizzes )

Marc Brierley, Stanford University Interaction Designerhttp://bugs.sakaiproject.org/confluence/display/SAM

Agenda

• Feature Overview– Focus on pedagogical flexibility using

assessment types

• Short Demo

• Futures

• Q&A

What are T&Q’s features:

• Assessment Authoring• Assessment Publishing• Assessment-Taking Management• Assessment Grading• Question Pools Management• Assessment-Type (formerly Template)

Management

Why “Assessments”?

• More than just test and quizzes

• For Example:– Problem Sets– Formative Assessments– Surveys

• The name “Tests and Quizzes” is a compromise

Organize & Publish Assessments

• Core (pending) Assessments for a course– Starting page for assessment authoring– Can be ex/imported in QTI XML format– When ready, published via Settings

• Published Assessments for student assessment-taking– When students respond, scores link appears– Can change release, due, retract dates

Question Types• Multiple Choice (single and multiple correct)• Multiple Choice Survey• True/False• Matching• Essay/Short Answer• Fill in the Blank• File Upload• Audio Recording** New for Sakai 2.2 release

Rich-Text & Attachments

• In section headers, question texts, selections for responses, and feedback

Pedagogical Flexibility

• Online test

• Self-study questions

• Homework, problem-sets

• Paper, code, or project submission

• Language drills

• Quick knowledge probes

• Surveys

• Others???

Assessment Types

• Create assessment “types” for different disciplines, different uses

• Hide complexity by selecting what which settings are visible to the instructor.

• Pre-set defaults by making setting choices for the type (e.g., Self-study always has immediate feedback)

Assessment Creation Workflow

Online Test

• Unique tests for each student– Randomized order of questions– Randomized draw from question pools

• Timed test-taking during access window– Auto-submit at end of timed period

• One submission only per student• Higher Security

– IP Addresses restricted– Secondary Password (proctored)

• No late submissions accepted• Scores transferred to gradebook

Self-Study Questions

• Immediate feedback

• Random access to Questions – Table of contents– Marked for review list

• No record of score in gradebook

• No due date; always available to student

• Unlimited submissions allowed

Homework/Problem Sets

• Published with multiple release and due dates for different sections

• Multiple methods for handling late submissions• Students can save work during assignment

period.• Auto-grading with methods for diagnosing

learning problems • Quick-reviews of student aggregate performance

by instructors viewing histograms, statistics.• Pools for organizing questions for reuse

Essay and Project Submission

• Use file upload question type for submitting any type of document

• Grader can download assignment; upload marked-up file to return to student

• Grades are recorded in gradebook

Question Pools

• Organize questions into multiple subpools

• Author Questions from within Pool Mgr.

• Randomly draw from pools for an assessment

Demo

2.3 Highlights

• Attachment of files and URLs to• Audio Question Type w/ signed certificate• Bulk download of student files• Filtered view in grading• Page navigator and search capability for the

Grading pages• New question type: Numeric Responses

– By Diego del Blanco Orobitg (Universidad Politécnica de Valencia)

2.4 Roadmap and beyond

• 2.4– email student feedback from grading screens

SAK-7094– performance enhancements to student

assessment taking (delivery)– bug fixes

• Beyond?...stay for the BOF

Questions?

Contacts

Marc Brierley, Stanford University T&Q Lead Designer marc.brierley@stanford.edu

Lydia Li, Stanford University T&Q Project Manager lydial@stanford.edu

Homepage:http://bugs.sakaiproject.org/confluence/display/SAM

Where did the design originate?

• Formative Assessment (Homework)– Stanford Assignment Tool for Human Biology– CourseWork CMS with Language Tools

(Andrew W. Mellon Foundation funded)

– OKI Tools Workshop to Specify Functionality(Andrew W. Mellon Foundation funded)

– OKI Formative Assessment Tool(Andrew W. Mellon Foundation funded)

• Summative Assessment (High-Stakes tests)– OnCourse Quiz Tool

Where did the design originate?

• OKI Tools Workshop– Many schools met to define needs for a CMS– One group specified the needs for an

Assessment Tool– Several hundred functional requirements

identified

Where did the design originate?

• Assignment and Assessment Manager – Supports teaching and learning in

large lecture science courses and in language instruction

– This OKI Tool was developed at Stanford– Andrew W. Mellon Foundation funded

Where did the design originate?

• AAM design combined with OnCourse Quiz tool design “Navigo” (2003)

• Stanford and Indiana join Sakai project, Stanford rewrites tool to Sakai Style Guide TPP, tool renamed SAMigo (2004-present)