Upload
jovany-hard
View
221
Download
6
Tags:
Embed Size (px)
Citation preview
HERDING JAVELINAS :CoursEval Implementation at a Large Multi-Campus University
INTRODUCTION
Arizona State University, with more than 70,000 students in 15 colleges, has adopted CoursEval as our sole university-supported
course evaluation system.
Over the course of a single academic year, we expanded the use of CoursEval from two
colleges to nearly all online and on-ground courses.
INTRODUCTION
We will describe how we: achieved buy-in from administration,
faculty, staff, and students; retired our legacy paper and online
evaluation systems; balanced decentralized college-level
ownership with centralized facilitation, and saw considerable cost-savings, with no major problems.
HERDING JAVELINASPaper evaluations were used for in-person classes for most of ASU
Strengths Units had control over survey design and content Familiar system, little training involved
Weaknesses Required considerable consumption of paper,
printing, and personnel time
o ASU commitment to reduce our carbon footprint Less than ideal for data retention, historical
tracking, etc.
HERDING JAVELINAS
Online Evaluation Legacy System Strengths:
Centralized administration
Reporting convenient for units
Legacy System Weaknesses
1. Used separate systems for survey administration and analysis/reporting.
2. Single survey for all units - did not meet needs of some (and did not match survey in use for on-ground sections). This was an issue for promotion/tenure.
3. No comparative/historical data options.
Legacy System Weaknesses (cont’d)
4. Access control required SQL programming by Associate Director to update database. Time consuming and could not delegate to staff.
5. Not available for on-ground courses, despite considerable demand from units.
6. Limited to single administration per traditional semester. Did not accommodate non-traditional course schedules.
7. Data analysis and reporting were time-consuming. Reports not available for several weeks after close of survey.
Our Requirements:
a. Online surveys for both online and in-person courses
b. Units can use their own survey(s)
c. Flexible administration schedules, controlled by units
d. Results available immediately
e. Release of reports (and report content) determined by units
SEARCHING FOR A SOLUTION
Our Requirements (cont’d):
f. Instructors, chairs, & dept. support staff have access to results
g. Reports available online for both current & future access/review
h. Flexible reporting format
i. Support comparisons across instructors & across time
j. Support co-taught sections
SEARCHING FOR A SOLUTION
We explored these options :
a. Re-design homegrown legacy system
b. Use one of the homegrown systems in use by ASU’s Engineering or Business schools
c. Identify a commercial system
10
DECISION:PURCHASE COMMERCIAL SYSTEM
Engineering & Business systems could not be easily modified
Re-design of legacy system would require considerable work by developers
Mary Lou Fulton Teachers College used CoursEval for years with good results. They did demo for us, and we were impressed with features
PILOT TEST
Attended Sept. 2010 Courseval User Group meeting to determine whether CoursEval would meet ASU’s needs
Worked with Brian Hopewell during the meeting to negotiate a price for a limited pilot
Fall 2010 Used CoursEval for online courses in all colleges
except Engineering & Business; included in-person courses for Technology
Participation was voluntary, but many used it
Spring 2011
CoursEval available for all online and in-person sections
Participation is still voluntary, but most made transition
TRANSITION
SPRING 2010
Scanned
119,038
Surveys
SPRING 2011
Scanned
49,336
Surveys
TRAININGFall 2010
a. Trained one user in each college to create survey items & responses, and assemble surveys. After surveys closed, trained them to design and release reports.
b. Training scheduled in one on one sessions - via WebEx, or in-person.
c. Coached users through cleaning course/instructor data from PeopleSoft before our staff imported into CoursEval
d. I handled entire administration: e-mails, reminders, & questions from faculty & students
DISTRIBUTING RESPONSIBILITIES
Winter 2010
I transferred responsibility for e-mails to college users – sending them detailed instructions (including screenshots)
I continued to manage PeopleSoft queries and data imports
Spring 2011 I was instructed to fully transfer
responsibility to the units
Several users realized that there was too much work for one person, and wanted me to train others in their areas
DISTRIBUTING RESPONSIBILITIES
SPRING 2011 CoursEval agreed to send Michele to ASU for a two-day Train the Trainer workshop. Approximately 40 users from across the university registered for the workshop.
Michele trained them on:
Creating items & responses
Building surveys
Data imports
Survey administration
Reports
SPRING TRAINING - 2011
We also taught the group how to run the PeopleSoft query for course, instructor, and student data, how to clean the data, and how to do imports.
We obtained approval for me to send a common e-mail invitation to students to reduce the number of student e-mails.
Established the ASU CoursEval User Group, so that users from across the university can support one another.
Invitations
Responses
Rate
Spring 2010
29,998 14,041 47%
Spring 2011
140,543 75,701 54%
SPRING 2011
We thought we had our pigs in a row! But then…
SPRING 2011
Shortly after the spring training, we lost several of our power users!
SUMMER 2011
More Challenges Some trained users left on vacation
during open surveys
More staff turnover
Proposed Solutions Train more people?
Return to centralized model?
FALL 2011
Stay Tuned …We are in discussions with ASU leadership about
how to balance unit-level autonomy with centralized support and continuity.
Wanda K. BakerAssociate Director for Assessment & Information TechnologyUniversity Office of Evaluation & Educational EffectivenessArizona State UniversityTempe, [email protected]