Embed Size (px)
Citation preview
Enhancing OSP for Programmatic and
Institutional Assessment
Lynn Ward and John Gosney
Indiana University
IUPUI: Institutional Profile
• Founded 1969 with a strong local mission• Blended campus• Metropolitan research university• 20+ schools (15 with professional/pre-
professional foci)• Commuter campus, ~30,000 students
(~20,000 undergraduates)
Approach to General Education
• Early 1990s - General Education based in the schools—distributive model
• Accreditation prompts internal reflection• Campus mandate for change, specifically a
centrally coordinated approach and specific learning outcomes for general education
• 1998 – Campus adopts a competency or ability-based model
3
Principles of Undergraduate Learning (PULs)
• Core Skills – written and oral communication
– ability to comprehend, interpret, and analyze texts
– analytical (quantitative reasoning)
– Information and technological literacy
• Critical Thinking• Integration and Application of Knowledge• Intellectual Depth, Breadth, and Adaptiveness• Understanding Society and Culture• Values and Ethics
Assessment Needs• Document and demonstrate the effectiveness of
IUPUI’s approach to general education (HLCNCA and ICHE)
• Document student achievement in programs subject to specialized accreditation (Education, Engineering, Visual Communications, etc.)
• Standard reports that aggregate and summarize assessment data across courses and programs
• Filter and group on demographic and academic criteria
5
Original Vision
6
Challenges• Site-centric nature of OSP tools; no way to easily
aggregate data across sites
• No tools to simplify management of very large sites
• Customization also makes it difficult to aggregate data; each department uses a different evaluation form and rating scale.
• No canned reports; every report requires an experienced XML programmer understands underlying data structures
• Academic programs more concerned with their own disciplinary outcomes than PULs
7
Current Vision: Phase 1: Goal/Outcome Linking
• Instructor or program administrator creates and publishes goal set; goal set becomes aggregation point
• Instructors in program can link any course assignment, matrix cell, or wizard page to one or multiple goals
• Students can attach examples of their work directly to one or multiple goals.
• Standardization of evaluation form elements makes it possible to aggregate and report data across courses and programs
Goal/Outcome Linking
EDUC 301
EDUC 401 PracticumPortfolio
Principles of Teacher Education
Filtering and Grouping
Secondary Education
Phase 2: Goal/Outcome MappingProgram
OutcomesProgram
Outcomes
Institutional Outcomes
Institutional Outcomes
Campus-level Aggregation via Mapping
IUPUI
Publishing and Linking Items to Matrices
13
Create and Publish Matrix
14
Manage Site Associations
15
Associate Site(s) with Matrix
16
Create Assignment in “Associated” Site
17
Link Assignment to Matrix Cell(s)
18
Student View of Linked Assignment
19
Linked Assignment in Matrix Cell
20
Opened Assignment in Cell
21
Linkable Tools
• Assignments
• Matrices (cells can be linked to other cells)
• Wizards (pages can be linked to cells)
22
Other OSPEnhancements
The Problem
Matrix authors must select forms and evaluators in each cell, even when the same choices are used in every cell.
24
The Solution: Matrix Defaults
25
Use All Defaults
26
Override All Defaults
27
The Problem
• No workflow support for reviewers (providers of formative feedback)
• Students cannot solicit feedback from peers, advisors, etc.
• No way for individuals who are not CIG members to provide feedback
28
Reviewer Workflow
• Submit for Review button
• Email Notification• Eventually …
– Recipient notification– Visual indicator of new
feedback
29
Student Initiated Feedback
30
External Reviewers
• Email notification provides direct link to cell (Sakai UserID and Password is required)
• Eventually …– Reviewer dashboard in My Workspace to
aggregate pending feedback requests
31
The Problem
Submitted artifacts are forever frozen in My Workspace >Resources, which precludes users from reorganizing files and folders at a later time.
Also, students do not understand why some files in their personal resources cannot be moved or deleted.
32
Solution: Attachment Helper
33
Direct upload from local storage device.Direct upload from local storage device.
The Problem
Users in evaluator (or reviewer) role can open all cells in a users matrix, even if not selected as an evaluator for the cell.
More granular access control is needed to support range of implementation scenarios from highly sensitive and secure to open and collaborative.
34
The Solution
• Cells can only opened by the designated evals/reviewers
• Revision of matrix permissions to provide much greater flexibility and granularity in progress
35
The Problem
Cells that have been evaluated and returned for additional evidence or other modifications look just like cells that have never been submitted.
36
The Solution: Returned Status
37
What’s Next?
38
For Fall 2008
• Standardized evaluation form and reports • Enhanced evaluator workflow and dashboard• Reviewer dashboard• Auto-population of portfolio sites based on
membership of associated course sites• Linking to wizard pages, and overall wizard
improvement
39
