A Comparison of Electronic Portfolio Systems Used for Individual and Program Assessment David Wicks,...

A Comparison of Electronic A Comparison of Electronic Portfolio Systems Used for Portfolio Systems Used for

Individual and Program Individual and Program AssessmentAssessment

A Comparison of Electronic A Comparison of Electronic Portfolio Systems Used for Portfolio Systems Used for

Individual and Program Individual and Program AssessmentAssessment

David Wicks, Seattle Pacific University – dwicks@spu.edu Cris Guenter, California State University, Chico - cguenter@csuchico.edu Jane Moore, National-Louis University - jmoore@nl.edu

Background: Physical and theoretical Background: Physical and theoretical qualities inherent to portfoliosqualities inherent to portfolios

Type – either working or showcase Organization – chaotic or standards driven Type of student artifacts – text or media based Feedback/assessment – summative and formative Nature –static or dynamic and evolving Heuristic processes – simple to complex Context – student or faculty provided Delivery mode – paper, e-portfolio, web-portfolio

(Love, McKean & Gathercoal, 2004)

Five Levels ofFive Levels ofPortfolio MaturationPortfolio Maturation

(Love, McKean & Gathercoal, 2004)

I. ScrapbookII. Curriculum vitaeIII. Curriculum Collaboration between

student and facultyIV. Authentic evidence as the authoritative

evidence for assessment, evaluation, and reporting

V. Mentoring leading to mastery

Three systems discussedThree systems discussed

School Portfolio System Server

SPU Chalk & Wire ASP (hosted)

CSU Chico

STEPS Home grown

NLU LiveText ASP (hosted)

Evaluating portfolio systemsEvaluating portfolio systems

1. Ability to function as a program audit tool2. Authoring flexibility for students3. Ability to be used by faculty as an assessment

tool. 4. Security and maintenance of user data5. Cost effectiveness6. Accessibility for students/faculty

with disabilities.

1. Ability to function as 1. Ability to function as a program audit toola program audit tool

a. Ability to customize system to align with institutional distinctives.

b. Incorporation of assessment rubrics allowing instructors to provide consistent feedback to students.

c. Ability to aggregate and disaggregate assessment data.

d. Inclusion of a robust reporting tool to assist with data-driven decision making.

Example from SPU

2. Ability to be used as 2. Ability to be used as an authoring tool by studentsan authoring tool by students

a. Ability to use institutionally created templates b. Access to and amount of virtual drive space for storage

of various artifact file types.c. Ability to link artifacts to multiple competencies.d. Ability to edit artifact information after uploading.e. Capacity to create multiple portfolios with one account.f. Ease of submitting work for assessment.g. Ease of reviewing work that has been assessed to help

clarify which competencies still need to be addressed.h. Portability of finished portfolio.

Example from National-Louis

3. Ability to be used by faculty 3. Ability to be used by faculty as an assessment tool. as an assessment tool.

a. Provides mechanism to notify faculty when a submission is ready for assessment.

b. Ability to use scoring rubric while viewing artifact(s) and reflection (on screen at same time).

c. Ability to add comments for each rubric criterion.d. Provides a mechanism to automatically notify

students when assessment completed.e. Provides a view/report that allows faculty to

quickly assess whether student has demonstrated competency on all standards.

Example from CSU Chico

Example of Artifact

Example of Rubric

4. Ability of portfolio system to 4. Ability of portfolio system to adequately security and adequately security and

maintenance of user data.maintenance of user data.

a. Follows best practices for backing up data.b. Adequately protects against viruses

and hacking.c. Provides adequate "up-time" for

institution’s needs.d. Privacy of individual student work sharing

only items student decides to share.

5. Evaluation of portfolio system in 5. Evaluation of portfolio system in relationship to cost effectiveness.relationship to cost effectiveness.

a. Minimal impact on system resources.

b. Costs can be tracked and charged to individual users during early adoption phase.

School Portfolio System

Student cost

SPU Chalk & Wire $100

CSU Chico STEPS $55

NLU LiveText $80

6. System’s ability to address 6. System’s ability to address accessibility needs of accessibility needs of

students/faculty with disabilities.students/faculty with disabilities.

a. Incorporation of Section 508 standards.

b. Ability to accommodate multiple languages in single system.

Wish listWish list

Example of Student ArtifactExample of Student Artifact

Return

Student Authoring ExampleStudent Authoring Example

Return

Faculty Assessment ExampleFaculty Assessment Example

Return

Program Audit ToolProgram Audit Tool

Return

Return

Example of RubricExample of Rubric