CLEW!Survey!2011!!mediawikibe.uwindsor.ca › wiki › images › c › c1 › CLEWSurveyFinal.pdf · 2017-06-13 · CLEW!Survey!2011!! Students,!Instructors,!andStaff! Prepared!by!LorieStolarchuk,!LearningTechnologies!Educational!Consultant!

CLEW Survey 2011 Students, Instructors, and Staff

Prepared by Lorie Stolarchuk, Learning Technologies Educational Consultant Nick Baker, Learning Specialist

Centre for Teaching and Learning University of Windsor

September, 2012

Centre for Teaching and Learning CLEW Survey

2012

2 | P a g e

The following employees of the CTL, Leddy Library, and the Office of Institutional Analysis contributed to this report: Lorie Stolarchuk, Learning Technologies Educational Consultant, CTL Nick Baker, Learning Specialist, CTL Erika Kustra, Director, Teaching and Learning Development, CTL Daniel Edelstein, Academic Data Specialist, Information Services Department, Leddy Library Robyn Nease, Institutional Analysis For more information, please contact: Centre for Teaching and Learning Lambton Tower, Basement and 2nd Floor 401 Sunset Avenue, Windsor, Ontario N9B 3P4 (519) 253-‐3000 ext. 3090 [email protected] | ctl.uwindsor.ca


2012

3 | P a g e

TABLE OF CONTENTS EXECUTIVE SUMMARY -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 6

CLEW: COLLABORATION AND LEARNING ENVIRONMENT WINDSOR -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 7

INTRODUCTION -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 7 CLEW OVERVIEW -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 7

CLEW and Sakai -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 8 Infrastructure -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 8 Staffing -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 9

GOVERNANCE -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 10 LMS Steering Committee -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 10 LMS Advisory Committee -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 10 CLEW Team -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 10 CLEW Implementation Team -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 10 CLEW Technical Team -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 10

LEARNING MANAGEMENT SYSTEMS IN ONTARIO -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 11

CLEW SURVEY -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 12

METHODOLOGY -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 12 SURVEY DEMOGRAPHICS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 12

Instructors and staff -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 12 Faculty or Department -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 12 Years of teaching experience -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 13 Gender -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 14 Self-‐Reported computing expertise level -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 15 Students -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 16

WHO IS USING CLEW? -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 18 COURSE SITES -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 18 PROJECT SITES -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 21 STUDENT USE -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 21

HOW IS CLEW USED? -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 23

CLEW TOOLS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 23 NON-‐CLEW TOOLS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 25

CLEW SUPPORT: USER PERCEPTIONS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 26

TRAINING: INSTRUCTORS AND ASSISTANTS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 26 Workshop Attendance -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 26 Workshop Format -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 27 CLEW Help Wiki -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 27

SOURCES OF SUPPORT: RATINGS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 27 TECHNICAL SUPPORT -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 29


2012

4 | P a g e

KEY QUALITATIVE DATA -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 32

BEST OR MOST USEFUL CLEW FEATURES -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 32 Instructors/Staff -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 32 Students -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 32

WORST OR LEAST USEFUL CLEW FEATURES -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 33 Instructors/Staff -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 33 Students -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 34

MOST IMPORTANT IMPROVEMENTS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 35 Instructors/Staff -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 35 Students -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 37

WHAT DO WE NEED IN AN LMS? -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 38 Instructors/Staff -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 38 Students -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 39

WHAT WOULD MAKE CLEW EASIER FOR YOU TO USE? -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 40 Instructors/Staff -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 40 Students -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 41

KEY QUANTITATIVE STATISTICS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 42

EASE OF USE -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 42 HOW DOES CLEW COMPARE TO OTHER LMS’ RESPONDENTS HAVE USED? -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 44

Instructors/Staff -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 44 Students -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 44

QUANTITATIVE STATISTICAL ANALYSIS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 46

INSTRUCTOR/STAFF DATA -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 46 Correlational data -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 46 Likert-‐Scale question data Summary -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 46 t-‐Tests -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 46

STUDENT DATA -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 47 Correlational data -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 47 Summary of Likert-‐Scale question data -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 47 t-‐Tests -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 47 Modeling and Predicting Ease of Use -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 48

LMS STRENGTHS AND WEAKNESSES -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 48

KEY THEMES -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 50

WOULD LIKE CLEW TO . . . -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 52

RECOMMENDATIONS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 53

SYSTEM -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 53 USABILITY IMPROVEMENTS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 55 SUPPORT -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 55 COMMUNICATIONS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 57 FORMAL PROCESSES FOR REVIEWING FEATURE REQUESTS GROUP -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 58


2012

5 | P a g e

CONCLUSION -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 59

APPENDICES -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 60

APPENDIX 1 -‐ CLEW SURVEY – INSTRUCTORS AND STAFF QUESTIONS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 60 DEMOGRAPHICS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 66 APPENDIX 2 – CLEW SURVEY – STUDENT QUESTIONS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 68 DEMOGRAPHICS -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 73 APPENDIX 3: SUPPLEMENTARY CHARTS AND TABLES -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 74

LMS Steering Committee -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 74 Instructor distribution by Faculty -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 75 Faculty / sessional gender fistribution Fall 2011 -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 76 Instructor / staff CLEW tool use -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 77 Student CLEW tool use -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 78 Students – Tools used for group work -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 79

REFERENCES -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 80


2012

6 | P a g e

Executive Summary The data contained in this report is derived from a survey administered to University of Windsor faculty, staff, and students in November 2011. Data is supplemented by previous CLEW-‐related surveys from 2008 and 2009, as well as institutional and systems data where relevant. The high response rate (1024 students and 292 instructors/staff) to this survey reflects substantial interest in CLEW as a fundamentally important system on campus. The majority of instructors/staff (69%) rated CLEW either “Good” or “Excellent” in terms of stability, reliability and support for technical issues. Improving these aspects of the system was an area of focus of 2007-‐2009 and these figures indicate this has been achieved. The majority of instructors (92%) also indicated they would recommend using CLEW to their colleagues and 87% indicated that continuing to learn (about CLEW tools) was beneficial to them. Responses indicate that instructors/staff are most frequently using “push” or “broadcasting” tools such as Announcements, Syllabus, and Resources. There has been a large increase in the use of assessment and grading tools over the last three years, with Assignments and Gradebook rated in the top 10. Users report satisfaction with CLEW for its “administrative” capabilities including functions such as managing class lists, online grading, posting and reading announcements, posting the course syllabus, storing lecture notes, and providing statistics for a course site; most see CLEW as having significant improvements over the previous learning management system, ViCKi. However, despite the majority of courses using CLEW to some extent, most students who responded are frustrated with inconsistent and low-‐level use of the system by instructors. Students and instructors alike also indicate frustration with the user interface, navigation, inconsistency from tool to tool, and speed of access to CLEW’s tools (load time). They also expect more sophisticated and easier-‐to-‐use collaborative tools for student-‐to-‐student and student-‐to-‐instructor interactions, greater flexibility in personalization, access via mobile devices, and indicate that more web 2.0 tools should be integrated with the system. Despite institutional statistics indicating that CLEW is relatively stable and reliable with high and improving uptime, there was significant frustration expressed in the qualitative comments about downtime, students and instructors not being able to access CLEW (particularly around important dates), and cross-‐platform compatibility issues. It is important to note that users may not always be able to distinguish “CLEW” related system issues from other campus systems that may impact the performance of CLEW. The most important improvements that instructors and students requested, and which they perceived would make the system more useable for them, include a redesigned, more contemporary user interface, simpler navigation, and a simplified approach to workflow that requires far less steps (and therefore less time) to access and alter common functions. There was also a perceived lack of functionality in synchronous and asynchronous interactivity/communication tools, instructor-‐generated quizzing and testing, tools to support group-‐work, assessment and grading tools, and integration with other campus systems.

CLEW: Collaboration and Learning Environment Windsor INTRODUCTION Providing multiple ways for students to engage and interact with their instructors, peers, and other university staff, inside and outside of the classroom, is widely viewed as a key indicator of student success (Kuh, Kinzie, Schuh, & Whitt, 2005). Learning management systems (LMS) are personal learning environments – when used most effectively, they can impact student engagement by influencing how, when, and where students access course information and materials, and more importantly, connect with instructors and classmates. This call for a supportive learning environment is championed by University of Windsor students: according to a 2011 National Survey of Student Engagement (NSSE), two-‐thirds of students (67%) find that the University is substantially committed to providing an environment that is supportive of student learning. Now in its fifth year, uptake of the University of Windsor’s LMS has increased every year since its inception: in Fall 2011, 96% of students registered at the University were enrolled in at least one course that used the institution’s LMS, CLEW (Collaboration and Learning Environment Windsor) – this system is clearly a critical component of the learning environment at the University of Windsor. In November 2011, the CLEW Team administered a survey to students, staff, and faculty across campus in order to better assess CLEW’s effectiveness and usability as well as identify areas of improvement. This report summarizes the data collected from the CLEW survey and outlines a number of recommendations to help improve the LMS for all user groups across campus. Information gained from the survey will be used to enhance the learning management system in order to help support and provide an exceptional and supportive student and staff experience. CLEW OVERVIEW CLEW is used to support teaching and learning in face-‐to-‐face, distance education, and blended courses. Using CLEW, instructors can:

• Post lecture notes online and link to other online resources, • make class announcements, • guide learning through online lessons, • facilitate online asynchronous discussion, • accept assignments and provide feedback online, • display important dates and subscribe to an institutional academic calendar, • post an online syllabus, • integrate grades from assessment tools into the Gradebook, and • post and transfer grades to the campus eGrade system.

CLEW is also designed to support research and collaboration efforts. CLEW may be used to facilitate ad hoc collaboration, to administer a study, to collaborate on a grant, or to run a study group between individuals within one or several institutions.


2012

8 | P a g e

CLEW AND SAKAI CLEW is based on the open source software known as Sakai (otherwise known as a Collaboration and Learning Environment or CLE), which is developed by a global network of developers. This approach offers flexibility (in adoption and in a development approach), avoids vendor lock-‐in and associated license costs, and allows for customisation by internal developers. Partners involved in the Sakai project include Stanford, Indiana, Yale, Berkeley, Oxford, Cambridge and the University of Michigan. The University of Windsor was one of the first Canadian partners in the Sakai community. Globally, there are over 350 institutions that are currently using Sakai (Figure 1).

Figure 1. Institutions Using Sakai (Sakai, 2012)

INFRASTRUCTURE Operating CLEW involves using “Load Balancing” technology, and an “Application Server Cluster.” Load Balancing refers to technology that distributes workload across a computer cluster to avoid overloading any one server. A “cluster,” is a series of computers that work together as a single unit to achieve higher performance and efficient application delivery.


2012

9 | P a g e

At the University of Windsor, there are six Java Web Application Servers that form the Application Server Cluster. These servers are load balanced by two dedicated F5 servers, which form the F5 Load Balance Cluster. The single CLEW database (Oracle 11) holds all of the content and data that is presented in CLEW and is connected to the CLEW application server group (Figure 2). The tools that have been developed at the University of Windsor are housed on one of the CLEW servers based in ITS, called “CLEOSUN.” These include: Email Instructor, Content Viewer, and Quizzes & Tests.

Figure 2. CLEW Architecture

STAFFING The LMS is a joint project between the Centre for Teaching and Learning (CTL) and Information Technology Services (ITS). Two full-‐time staff are dedicated to CLEW and its administration: an LMS Programmer from ITS, who provides system administration and development, and an LMS Administrator from the CTL who provides end-‐user support and system administration, and performs course lifecycle tasks. The CTL’s Learning Technologies Educational Consultant provides training and end-‐user support and documentation, accounting for a 0.6 full-‐time equivalent (FTE) staff position. The CTL Application Programmer contributes to system development, accounting for a 0.2 FTE position. ITS also supports end-‐users through their Help Desk and other staff who have a part-‐time focus on administering Sakai systems. Additional members of the CLEW Team include the Assistant Director (ITS), the Director, Teaching and Learning Development (CTL), Manager, Teaching and Learning Technologies (CTL), and a


2012

10 | P a g e

Teaching and Learning Specialist (CTL). These members provide guidance, administrative support, budgetary control, and technical and pedagogical perspectives. GOVERNANCE LMS STEERING COMMITTEE The LMS Steering Committee is tasked with making all strategic decisions for CLEW. This committee is comprised of members of the CTL, ITS, the Office of the Registrar, and Leddy Library (Appendix 3), and is chaired by the Vice-‐Provost, Teaching and Learning. This group meets approximately every six weeks. LMS ADVISORY COMMITTEE Representatives from each faculty on campus, the CTL, ITS, University of Windsor Students’ Alliance, Graduate Students Society, Organization of Part-‐time Undergraduate Students, Leddy Library, Student Disability Services, Public Affairs and Communications, and Office of the Registrar sit on the LMS Advisory Committee. Several committee members who sit on the LMS Steering Committee (Chair, Director, Teaching and Learning Development; Manager, Learning Technologies; and the Assistant Director, ITS) also sit on the Advisory Committee. This committee provides feedback, communicates pertinent information to their constituents, and makes recommendations to the LMS Steering Committee concerning strategic decisions. This committee meets 5-‐6 times a year. CLEW TEAM The CLEW Team is comprised of members from the CTL and ITS, and meets weekly. Its role is to discuss development strategies, manage day-‐to-‐day issues, and provide recommendations to the LMS Steering Committee regarding issues such as system upgrades, major feature requests, or major system deficiencies. Members from CTL include the Manager, Learning Technologies (Chair); Director, Teaching and Learning Development; LMS Administrator; Learning Technologies Educational Consultant; and Teaching and Learning Specialist. Members from ITS include the Assistant Director, the LMS Programmer, and a HelpDesk representative. CLEW IMPLEMENTATION TEAM This team meets as needed to review budgetary issues, strategic priorities, and major system decisions. It is comprised of the Manager, Learning Technologies (CTL), the Director, Teaching and Learning Development (CTL), and the Assistant Director (ITS). CLEW TECHNICAL TEAM Members of the CLEW Team occasionally must access a wider circle of technical support in ITS when the need arises. Non-‐CLEW dedicated members, who include Database and Network Administrators, Application Programmers, Systems Programmers, an Assistant Director, and any other staff, may be consulted as necessary.


2012

11 | P a g e

LEARNING MANAGEMENT SYSTEMS IN ONTARIO Several different learning management systems are used at universities throughout Ontario (Figure 3). In April 2012, Contact North released a six-‐part online series examining LMS’ across Canada. Open-‐source software accounts for 32% of the market share in Ontario (e.g., Instructure, Moodle, and Sakai), with the balance of the distribution falling to proprietary software. According to this study, Moodle has 40% of the market share in the rest of Canada (largely due to its popularity in British Columbia), and open-‐source software accounts for 47% of the market share overall. Proprietary software, which includes Blackboard, WebCT, Desire2Learn, Angel, First Class, and in-‐house developed solutions, accounts for 53% of the distribution. Sakai, the software that CLEW is based upon, is also supported at Brock University, and most recently, the University of Western Ontario.

Figure 3: Ontario Universities LMS Market Share (Contact North, 2012)

Instructure, 5%

Sakai, 9%

Desire2Learn, 27%

WebCT, 27%

Moodle, 18%

Blackboard, 14%

Instructure

Sakai

Desire2Learn

WebCT

Moodle

Blackboard


2012

12 | P a g e

CLEW Survey METHODOLOGY The CLEW Survey was administered in November 2011 across campus using an anonymous online survey tool called FluidSurveys (Appendices 1 and 2). Faculty, staff, sessional instructors, and students were invited to participate by email. Two hundred ninety-‐two (292) University staff (12.9%) and 1024 students (6.4% response rate) responded to the survey. The survey was available online for two weeks. The only mandatory question asked whether participants agreed to complete the survey or not. Only completed responses are included in this report. As a result, not all questions were addressed by all respondents. The study used a mixed methods approach, with both quantitative and qualitative data. Quantitative data was analyzed descriptively, and statistically, where appropriate. Descriptive statistics were generated and correlation analysis, t-‐Tests and regression analyses were performed on the quantitative data where possible using SPSS software. The surveys to both staff and students also employed unlimited, open response questions that provided a very large volume of free-‐form data analysed for themes and frequently used terms. Leximancer software was used to perform thematic, frequency, proximity, and path analysis of qualitative data, while QDataMiner and WordStat software was used to generate cluster analyses (agglomerative clustering and dendrograms, as well as bi-‐plots and tri-‐plots of similarity scores). Manual verification of themes was also completed. Additional data was collected from other institutional resources, including the CLEW database, the Office of Institutional Analysis, and the Office of the Registrar. This study had Research Ethics Board (REB) approval (REB Number 29535). Questions that resulted in statistically insignificant results were omitted from this report but data is available upon request. Campus-‐wide CLEW surveys were also conducted in Fall 2007 (n=93), Winter 2008 (n=39), Fall 2008 (n=98), and Winter 2009 (n=53) and were referred to for some question analysis where appropriate. A previous survey (Winter 2009) was re-‐designed to better align with the Sakai Community Survey known as Multi-‐Institutional Sakai Initiative (MISI). Data from the MISI was collected from participating institutions in 2009 and 2010, but due to a limitation in resources at the University of Michigan, MISI has not been updated since that time. Feedback from members of the LMS Advisory Committee was adopted in the design of this survey. The current survey was specifically designed to produce much more qualitative data than the previous ones. SURVEY DEMOGRAPHICS INSTRUCTORS AND STAFF Faculty or Department The response rate from the instructor/staff population was reasonably representative of the population. The highest number of responses came from instructors and staff in the Faculty of Arts and Social Sciences (FASS), followed by the Faculties of Science, Nursing, and Engineering (Table 1). The highest


2012

13 | P a g e

percentage of responses from faculty members within a single department came from the Faculty of Human Kinetics (65.4%), and the lowest percentage from the Faculty of Education (17.4%). A relatively high number of instructor/staff responses identified themselves as “Other” (24.8%). Additional instructor/staff respondents came from Student Services, the CTL, Finance, SWOMEN, Bookstore, Health & Safety, Planning Design & Construction, Athletics and Recreation Services, Social Work, Centre for English Language Development, English Language Improvement Program, ITS, Campus Police, Office of Human Rights, Equity & Accessibility, Residence, Centre for Executive and Professional Education, Centre for Smart Community Innovation, and Administration. Staff positions may not always be associated with CLEW course sites, but frequently make use of CLEW Project sites. For this question instructors/staff were asked to check all options that applied, therefore the percentage is greater than 100 as many staff members are in positions associated with more than one Faculty or department. A more detailed table containing instructor distribution by Faculty is available in Appendix 3, Tables 20 and 21. Table 1: Instructor Responses by Faculty1 Faculty Total Number of

Instructors on Campus2

Percentage by Faculty

Total Number of Survey

Responses

Percentage by Faculty

Response rate

FASS 302 33.6 80 33.6 26.5 Science 150 16.7 31 13.0 20.6 Nursing 99 11.0 30 12.6 30.3 Engineering 80 8.9 23 9.7 28.8 Education 115 12.8 20 8.4 17.4 Human Kinetics 26 2.9 17 7.1 65.4 Business 69 7.7 16 6.7 23.2 Law 55 6.1 10 3.6 18.2 Interfaculty programs

N/A N/A 1 0.4 N/A

Graduate Studies

N/A N/A 10 3.6 N/A

Total 899 238

YEARS OF TEACHING EXPERIENCE Instructors who responded ranged in experience from one year or less to over 30 years, with the majority of responses falling within 6-‐20 years of experience range. There appears to be a shift in the years of teaching experience from respondents from when CLEW was first introduced in 2007 to 2011. 1 The table excludes the “Other” option, which accounts for 24.8% of the total responses (59); and statistics for Graduate Studies and Interfaculty, as those faculty members are listed under their home. 2 The Office of Institutional Analysis provided the total number of instructors employed on campus. The number of survey responses for individual questions varies throughout the report as questions were not mandatory.


2012

14 | P a g e

In the 2007 and 2008 surveys, those with 5 years or less experience (over 50%) has shifted to the 6-‐20 years of experience in both the 2009 and 2011 surveys (Table 2). Respondents identifying themselves as having 11-‐20 years of experienced has increased from 8% (2008) to 24%, and the newer instructors with 1-‐5 years of experience declined from 45% (2008) to 22% (2011). This may have been influenced in part by the reduction in faculty hires, which began approximately in 2009. Table 2. Years of Teaching Experience in Higher Education (2007-‐9 and 2011) Fall 2007 Fall 2008 Winter 2009 Fall 2011

1 year or less 20% 6% 2% 7% 1-‐5 years 38% 45% 26% 22% 6-‐10 years 14% 18% 28% 28% 11-‐20 years 23% 8% 19% 24% 21-‐30 years * N/A 18% 19% 13% More than 30 years ** N/A ** N/A ** N/A 3% Prefer not to answer

** N/A ** N/A ** N/A 3%

Number of Respondents

93 98 53 286

* Category combined above ** Statistic not collected

GENDER The 2011 survey received a higher response rate from female instructors/staff (60%) than their male counterparts (36.5%) See Table 3 for a breakdown. This is only slightly different to the actual campus instructor/staff population, which is 57.3% female and 42.7% male. See Appendix 3, Table 21, for the distribution of male/female instructors by designation (faculty/sessional), and staff by full-‐time/part-‐time. Table 3. Gender Distribution on Campus and in the CLEW Survey (Fall 2011)3 Response Number of

Instructors/Staff on Campus

Percentage

Number of Survey

Responses

Percentage

Male 967 42.7% 105 36.5% Female 1299 57.3%

173 60%

Prefer not to answer -‐-‐ -‐-‐ 10 3.5%

Total 2266 100% 288

100&

3 The Office of Institutional Analysis provided the total number of male/female instructors/staff employed on campus in Fall 2011. The number of survey responses for individual questions varies throughout the report as questions were not mandatory.


2012

15 | P a g e

SELF-‐REPORTED COMPUTING EXPERTISE LEVEL INSTRUCTORS

Over the last four surveys, instructors and staff were asked to report their “expertise with computers” as generally intermediate to advanced (Table 4). The 2011 respondents’’ expertise was somewhat similar compared to the 2007, 2008, and 2009 cohort, except the intermediate users dropped from 65.3% to 53% and novice users increased from 2% to 7% in 2011. Those who preferred not to answer made up the additional 5% difference. Table 4. Self-‐reported Computing Experience Comparison (2007-‐9 and 2011) Fall 2007 Fall 2008 Winter 2009 Fall 2011

Novice 5% 2% 4% 7% Intermediate 67% 65% 66% 53% Advanced 28% 33% 30% 35% Prefer not to answer *N/A *N/A *N/A 5% Total 93 98 53 287 *Statistic not collected INSTRUCTOR/STAFF COMPUTER EXPERTISE COMPARED TO STUDENTS Instructors self-‐identified as having a higher percentage of advanced computer expertise than students (35% to 22%). Students indicated having a higher percentage of intermediate expertise than instructors (68% to 53%) See Figure 4 for a comparison.

7%

53%

35%

5% 10%

68%

22%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Novice Intermediate Advanced Prefer not to answer

Percen

tage

Perceived Level of Experese

Instructors/Staff Students


2012

16 | P a g e

Figure 4. Comparison of Instructor/Staff and Student Perceptions of Computer Expertise, Fall 2011 STUDENTS BY FACULTY

Students from all faculties responded to the survey (Table 5). FASS students provided the greatest percentage of total survey responses (consistent with the proportional size of their population at 40% of enrolments), followed by Science, Business, and Engineering. Representation was consistent with total student population, except for Graduate Studies, which had a lower response rate (4.3%) than in the total student population (11.4%). It is possible that students may have self-‐selected with their home faculty as opposed to Graduate Studies. Table 5. Student Responses by Faculty4 Faculty Number

of Students

on Campus

by Faculty (A)

Percentage (B) (A / E *100)

Number of Responses to Survey (C)

Percentage (D) (C/ E *100)

FASS 6344 39.9 414 40.6 Science 1814 11.4 147 14.4 Business 1687 10.6 105 10.3 Engineering 1002 6.3 88 8.6 Nursing 888 5.6 72 7.1 Human Kinetics 791 5.0 57 5.6 Education 624 3.8 63 6.2 Graduate Studies 1817 11.4 44 4.3 Law 616 3.9 40 3.9 Interfaculty programs

298 1.9 26 2.5 Total (E ) individuals)

15881 100 *1020 ** Total (Number ResponseResponses

Responses) **1056

* Number of unique individuals answering this question. ** Number of total responses, as students could self-‐select more than one Faculty.

4 The Office of Institutional Analysis provided the total number of students enrolled in Fall 2011 by faculty: http://web2.uwindsor.ca/general/info/pdf/Usis/Fall/All/2011/fall_2011_hc_fac.pdf


2012

17 | P a g e

YEAR OF STUDY

The majority of the student respondents identified as undergraduates (77.8% by semester (SM) 1-‐8), which is representative of the total student body (79.6%). See Table 6 for a breakdown of student responses by semester and program. Table 6. Student Responses by Semester (SM) or Professional Program (Fall 2011)5 Number of

Students by Level (A)

Percentage (B)

(A/E *100)

Number of Responses to Survey ©

Percentage(D)

(C/E *100)

Undergraduate SM 1-‐2 3579 22.5 177 17.3 Undergraduate SM 3-‐4 3113 19.6 192 18.8 Undergraduate SM 5-‐6 3305 20.8 204 20.0 Undergraduate SM 7-‐8 2647 16.7 222 21.7 Non-‐degree/certificate/*other

195 1.2 33 **N/A

Graduate – M1, M2, D1, D2 (non-‐Law or Education)

1803 11.4 130 **N/A

Professional Student (Education, Law)

1240 7.8 74 **N/A

Total (E) 15881 100.0 ***1022 100.0 * Survey had “other” category with 33 respondents. ** Percentage not included to avoid overestimating response rate as students may have self-‐selected multiple options. *** Total includes unique individuals, not responses,

5 The Office of Institutional Analysis provided the total number of students enrolled in Fall 2011 by faculty: http://web2.uwindsor.ca/general/info/pdf/Usis/Fall/All/2011/fall_2011_hc_fac.pdf. Variations may occur as survey was a self-‐report designation, and multiple options may have been selected (e.g., students may have selected Faculty of Science and GLIER as their faculty).


2012

18 | P a g e

Who is Using CLEW? The question -‐ Do you use CLEW or another Learning Management System in your teaching? – was a conditional point of entry into the more CLEW-‐specific questions in the survey (Table 7). Those who answered “Yes” (66%) were directed to the series of questions specific to CLEW use. Over one-‐third of the respondents indicated that they either do not use an LMS or have no opinion on using one. Those users were then directed to more general questions relating to the types of features needed for a LMS. Table 7. Instructors/Staff Use of LMS in Teaching Number of Responses Percentage Yes 194 66% No 72 25% No opinion 26 9% Totals 292 100% Almost half (48%) of the respondents who indicated that they had used an LMS, reported having used CLEW in seven or more courses (Table 8), where each offering is considered a new course (e.g., 03-‐60-‐100 F10 and 03-‐60-‐100 W11 are different courses). Table 8. Instructors: Within How Many Different Courses Have You Used CLEW? Number of courses Percentage None 5 2% 1-‐2 45 22% 3-‐6 54 27% 7-‐10 29 14% More than 10 68 34% Total Responses 201 100%

COURSE SITES CLEW adoption has grown considerably (Figure 5), and as of Fall 2011, there were a total of 1195 CLEW sites out of a possible 1932 course sections, representing 61.9% of all sections in that semester.


2012

19 | P a g e

Figure 5. Growth in the Number of CLEW Sites in Fall Semesters, 2007-‐2011

Some faculties use CLEW sites more consistently across their courses than others. In Fall 2011, the percentage of course sections with a CLEW site across faculties (from highest to lowest) was Nursing (95%), Business (90.8%), Human Kinetics (79.2%), Social Sciences (77.2%), Education (68.8%), Engineering (65.7%), Science (50%), Law (39%), Arts (36.6%), Interfaculty (36.4%), and GLIER (0%). See Figure 6 for a comparison. Nursing and Business had the highest proportion of courses with a CLEW site, while Interfaculty programs, Arts and Social Sciences, and Law all had approximately 1/3 courses using CLEW. GLIER did not have any course sections with a CLEW site in Fall 2011.

447

881

1029 1132

1195

0

200

400

600

800

1000

1200

1400

Fall 2007 Fall 2008 Fall 2009 Fall 2010 Fall 2011

Num

ber o

f Site

s

Semester


2012

20 | P a g e

Figure 6. Total Number of Course Sections Compared to CLEW Sections, Fall 20116

The number of instructors requesting CLEW sites has also grown rapidly from 284 in 2007 to 780 unique instructors in 2011, meaning 86% of all instructors were connected to a CLEW site in Fall 2011 (Figure 7).

6 Further statistics can be found at: http://cleo.uwindsor.ca/ctl/statsfactory/graph/clew-‐terms/


2012

21 | P a g e

Figure 7. Unique Instructors CLEW-‐Use: 2007-‐2011

PROJECT SITES In addition to course sites, the campus community also has access to project sites, which operate outside of the semester system and are intended for ongoing or special purpose collaborations and projects. Participants are added manually to these sites. As of Fall 2011, there were 282 project sites. STUDENT USE Most students have access to CLEW in one or more of their course sites (Figure 8). In Fall 2011, 96% of students enrolled had at least one course that featured a CLEW site.

0

100

200

300

400

500

600

700

800

900

F07 F08 F09 F10 F11

Num

ber o

f instructors

Year


2012

22 | P a g e

13300 13800

14498 14894

15254

10000

11000

12000

13000

14000

15000

16000

17000

Fall 2007 Fall 2008 Fall 2009 Fall 2010 Fall 2011

Num

ber o

f stude

nts

Fall Semester by year

Unique students in one or more CLEW sites by year (Fall semester only)

Total students

Figure 8. Unique Students in one or More CLEW Sites Compared to all Students Enrolled by Year: 2007-‐2011 (Fall Semester)7

7 Unique students in CLEW data was retrieved from CLEW production data, 2009-‐2012, State of the LMS report, lms-‐state-‐2007-‐11-‐02-‐00-‐25-‐04.txt. The Office of Institutional Analysis provided the total number of enrolled students. Due to statistical gathering methods available at the time, the data for number of students in the LMS Fall 2008 has been estimated.


2012

23 | P a g e

Every Course

Frequently

Sometmes Rarely

Never

Don't Know

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Percen

tage of R

espo

ndan

ts

CLEW Tools

Don't Know

Never

Rarely

Sometmes

Frequently

Every Course

How is CLEW Used? CLEW TOOLS Instructors/staff and students were asked: Which CLEW tools have you used in your courses? The most commonly used tools for both groups included those of a “broadcasting” nature, which facilitate one-‐way communication from instructor to student (Syllabus, Announcements, Resources, Lessons; Figure 9). The Lessons tool was perceived to have been used more by students than instructors/staff (Figure 10), followed by Assignments, Gradebook, Email Instructor, and Feedback. Instructors/staff believed that they used their chosen tools in every course, although students disagreed, which suggests that students may be noticing inconsistencies between tool deployment within courses. Syllabus, Announcements, Resources, Assignments, Gradebook, Lessons, and Feedback appeared in both lists (instructors/staff and students). Instructors/staff added the Roster (class list) and Site Stats tool in their top 10 list for administrative purposes, and students added Content Viewer (to view Resources) and Sections (for groups and appointments) in their top 10. For a total list of tool use for both groups, see Appendix 3, Tables 22 and 23.

Figure 9. Instructors’ and Students’ Top 10 CLEW Tools: Fall 2011


2012

24 | P a g e

Every course

Frequently

Sometmes

Rarely

Never

0

20

40

60

80

100

120

Percen

tage of R

espo

ndan

ts

CLEW Tools

Never

Rarely

Sometmes

Frequently

Every course

Figure 10. Student Perceptions of Top 10 CLEW Tools Used by Instructors/Staff: 2008-‐9 & 2011

Comparing tool use from 2008-‐2011, overall there appears to be consistency with respect to using “broadcasting – one way communication” types of tools (i.e. Announcements) with slight growth in use of the Syllabus and Resources tools (Figure 11). There is noticeable growth in use of the Assignments and Gradebook tools. This was consistent with the growth in workshop training and support calls for those tools during this period. There was a slight decline in the use of the Feedback tool from 2008-‐2011. Roster and Site Stats were not reported on in the previous two surveys, so there is no comparison data.


2012

25 | P a g e

Figure 11. Top 10 Tools Instructors Use: -‐ 2008-‐9 & 20118

Non-‐CLEW Tools Several respondents reported using multiple non-‐CLEW tools for their courses. These included: Google Docs (19), instructor websites (14), WordPress (6), Drop Box.com (4), Twitter (3), Lotus Notes (2), and publisher’s sites such as: WileyPlus, Nelson, Pearson, and Harvard case studies. Thirty-‐seven respondents reported using other tools such as YouTube, Facebook, EasyChair, iLink, Blackboard Collaborate (now available across campus), Tungle, Doodle, Class Record, Wimba, Markbook, Top Hat Monocle, Sharepoint, LimeSurvey, FluidSurveys, Blogger, Yahoo Groups, Tegrity, and Electron server.

8 Roster and site stats were not measured in previous surveys. Sample sizes are: 2008-‐98; 2009-‐53; and 2011-‐80/

0

20

40

60

80

100

120

Percen

tage

Tools

2008

2009

2011


2012

26 | P a g e

CLEW Support: User Perceptions TRAINING: INSTRUCTORS AND ASSISTANTS WORKSHOP ATTENDANCE In 2009, CTL initiated an online registration system for workshops, including CLEW Workshops. Since then, approximately 300 unique individuals have attended CLEW training events. Several instructors and assistants have attended more than one type of workshop. Prior to 2009, training on CLEW was held at ITS in the Computer Centre. When asked the question: Is continuing to learn about CLEW tools beneficial to you? 87% of respondents said yes. CLEW Basics for Instructors and Assistants is the most popular workshop, with 56% of the respondents who reported attending workshops indicated having taken it (Figure 12). The Gradebook Essentials workshop (22%), New Faculty Orientation (13%), and CLEW Basics for Nursing Orientation workshop (13%) were also popular. In Winter 2012, a new workshop targeted for graduate and teaching assistant (GA/TA) training was introduced and will continue to be offered. Since students reported receiving most of their support from their instructor or GA/TA, this training is important to ensure that students receive quality instruction.

Figure 12. CLEW Workshops Attended by Respondents

0% 10% 20% 30% 40% 50% 60%

CLEW Basics for Instructors and Assistants

Gradebook Essentals

New Faculty Orientaton

Managing Student Work: Using Assignments and Drop Box

Nursing Orientaton

CLEW Basics for Project Sites

Name Your Topic / Personal Consultaton

Interactvity: Polls, Forums and Wiki

Gradebook, Feedback & Quizzes: Conductng and Recording Assessments


2012

27 | P a g e

WORKSHOP FORMAT The majority of the respondents (46%) indicated that they preferred small group training (2-‐6 participants). One-‐on-‐one (35%) and mid-‐sized group (22%) training followed (Table 9). The least popular option was for a large group (over 15) workshop. Eighteen percent of respondents indicated that they did not want any training. Some of the respondents who indicated that they were sessional instructors found it difficult to attend workshops offered during regular office hours. CLEW workshops are offered at various times during the day (since 2007) and evening (since 2009). Blackboard Collaborate (piloted in Fall 2011) for synchronous online virtual classrooms now offers possibilities to attend virtual workshops online during non-‐office hours. Table 9. Preferred Training Format (Multiple Selections Possible) # of responses % of responses

Do not want training 15 18%

One-‐on-‐one 29 35%

Small group (2-‐6 participants) 38 46%

Mid-‐size group (7-‐15 participants)

18 22%

Large group (15 or more) 4 5%

Total 82

CLEW HELP WIKI In response to a call for more self-‐training opportunities (also supported by qualitative data), an online searchable Wiki was launched in Fall 2011; this site replaced the previous CLEW information site. The ratings for the CLEW Help Wiki as a source of support for instructors/staff was quite favourable – the majority of respondents who had used the Wiki rated it “Excellent” or “Good” (Table 13). The CLEW Help Wiki was developed using the open-‐sourced software, MediaWiki, and in collaboration with Brock University and the University of Florida. The Wiki has a search function and uses the same software as Wikipedia, so many users are familiar with its look and feel. The link to the CLEW Help Wiki is found on every CLEW site near the bottom of the left menu as well as the gateway login page. After the Fall 2011 upgrade, help information was updated to reflect changes, including screenshots of specific tasks and links to other relevant articles. SOURCES OF SUPPORT: RATINGS Participants were asked: Identify and rate your sources of support for CLEW (Check all that apply). The choices were the same for both groups, except that the students had the following additional options: “Your Professor,” “Your GA/TA,” and “Online Videos” (Tables 13 and 14). The data showed that most


2012

28 | P a g e

instructors and students had not accessed the majority of support sources available to them, ranging from 31% who had not used Help Desk, to 69% who had not attended any sessions with the Learning Technologies Educational Consultant (LTEC). Of the instructors/staff who sought support, the most heavily accessed resource (69%) was “Help Desk” followed by “[email protected]” email support (54% combined – help desk etc), which is monitored by both the LMS Administrator and the LTEC. The “Your Comments” form was the least used option at 84% (Table 10). When support was sought by instructors/staff, they appeared to be mainly satisfied with the results. In the open-‐ended comments, several respondents indicated that they sought help from support sources on specific more issues. It was clear that there were several positive comments about the customized support received, most notably from Terry Collins, the LMS Administrator. Others noted effective and helpful support received from the Learning Technologies Educational Consultant and Help Desk staff. The only notable exception was that Help Desk was rated slightly lower at “Average” by 10% of respondents, where other support sources were rated at “Average” by 0-‐3 % of respondents. Table 10. Instructors/Staff Ratings of Sources of Support Not

Used Excellent Good Average Mediocre Poor

Help Desk 31% 30% 28% 10% 1% 0% [email protected] 46% 31% 20% 2% 1% 0% LMS Administrator 49% 41% 8% 1% 1% 0% CLEW Help Wiki 67% 14% 16% 2% 0% 1% “Ask for help” form 67% 10% 18% 3% 3% 0% LTEC9 69% 25% 5% 0% 1% 0% “Your Comments” form 84% 9% 4% 2% 1% 0%

The vast majority of students indicated that they did not use the available institutional supports. The majority of students who sought support went to their professor, and 50% of these students found the support received from “Good” to “Excellent.” The students rated their GA/TA as the second most frequent source of support with 33% rating their GA/TA as either “Good” or “Excellent” (Table 11). Table 11. Student Ratings of Sources of Support Not

Used Excellent Good Average Mediocre Poor

Your Professor 19% 13% 37% 20% 7% 3% Your GA/TA 39% 8% 25% 17% 7% 3% [email protected] 70% 7% 11% 7% 3% 1% Help Desk 71% 3% 9% 9% 6% 1% Online videos 72% 2% 9% 9% 5% 2%

9 LTEC: Learning Technology Educational Consultant


2012

29 | P a g e

“Your Comments” form 77% 2% 6% 8% 6% 1% “Ask for help” form 79% 1% 5% 7% 5% 2% Online CLEW Help wiki 81% 2% 4% 7% 4% 2% * LTEC 83% 1% 4% 8% 3% 1% LMS Administrator 84% 1% 4% 6% 4% 1%

TECHNICAL SUPPORT For the purpose of this survey, technical support is defined under four broad categories including “Stability – (server up-‐time)” (Figure 13); “Reliability – (does what it is supposed to do)” (Figure 14); “Usability – (how easy user interface is to use)” (Figure 15); and “Support – (for technical issues)” (Figure 16). Respondents provided ratings for these categories from the following options: “Excellent,” “Good,” “Average,” “Mediocre,” “Poor,” and “No Opinion.” Instructor/staff ratings reflect a consistent view across the categories – most ratings were either “Good” or “Excellent.” The students, however, do not seem to be as enthusiastic with the “Support (for technical issues)” category. They do not appear to access the support as compared to the other categories (Stability, Reliability, or Usability), as 30% selected the “No Opinion” compared to the 17% of instructors 17%. Also, only 38% students report that the “Support – (for technical issues)” is “Good” or “Excellent,” compared to the 69% of instructors. Additionally, they are more likely to report “Average” or “Mediocre” ratings for “Stability (Server up-‐time)” as compared to instructors / staff. As CLEW is so widely used, it is possible that students may attribute other online technical issues that they experience (from other campus supporting system including myUWindsor portal, Student Information System (SIS), email servers, etc.) as a CLEW issue, thus impacting their view on server stability.


2012

30 | P a g e

Figure 13. Stability Ratings (Server Up-‐Time): Instructors Compared To Student Perceptions

Figure 14. Reliability10 Ratings: Instructors Compared to Student Perceptions

10 Does the system do what it is supposed to do?

0

20

40

60

80

100

Percen

tage of R

espo

nses

Raengs

Instructors

Students

0

20

40

60

80

100

Percen

tage of R

espo

nses

Raengs

Instructors

Students


2012

31 | P a g e

Figure 15. Usability Ratings: Instructors Compared To Student Perceptions

Figure 16. Support Ratings (Technical Issues): Instructors Compared To Student Perceptions

0

20

40

60

80

100

Percen

tage of R

espo

nses

Raengs

Instructors

Students

0 10 20 30 40 50 60 70 80 90

100

Axis Title

Raengs

Instructors

Students


2012

32 | P a g e

KEY QUALITATIVE DATA This section summarizes open-‐ended comments by identifying major themes and observations. Leximancer software was used to perform thematic, frequency, proximity, and path analysis of qualitative data, while QDataMiner and WordStat software was used to generate cluster analyses (agglomerative clustering and dendrograms, as well as bi-‐plots and tri-‐plots of similarity scores); manual verification of themes was also completed. BEST OR MOST USEFUL CLEW FEATURES INSTRUCTORS/STAFF The vast majority of responses from both instructors/staff and students to the question “What do you think is the best or most useful aspect of CLEW” centred on CLEW being a central location or repository (one-‐stop location) of instructional materials (lecture notes, resources, student work, and grades), and a mass-‐communication tool (syllabus, announcements, and ability to email all students) (Figure 17). Instructors also indicated that CLEW was very useful for grading and managing assignments. Some instructors mentioned that CLEW was relatively easy to use and that it was useful to be able to track who had used the site (including when and where they went), the student view option, and a number of individual tools (Calendar, Discussions, Web Content, Forums, Gradebook, Wiki, and Upload/Download all Assignments).

Figure 17. Instructor/Staff: Best or Most Useful CLEW Features STUDENTS Students tended to see the strengths of CLEW in the same way as instructors, with the best aspects of the system relating to the ability to access course resources such as notes, PowerPoint slides, readings, and the ability to receive announcements and email (Figure 18). Students also commented that online access to the syllabus and readily available grades were useful. They primarily saw the best aspects of the system as relating to broadcasting communications and access to course resources.


2012

33 | P a g e

Figure 18. Students: Best or Most Useful CLEW Features WORST OR LEAST USEFUL CLEW FEATURES INSTRUCTORS/STAFF Instructors/staff provided a great deal of information about the aspects of the system they felt were the least useful. The majority of comments clustered around the tools for grading and handling assignments, as well as the user interface (Figure 19). They made particular comments about the amount of time and the number of clicks required to achieve the desired outcomes, as well as the cluttered design of the interface in a number of tools (especially Lessons, Resources, and Discussions/Forums). There were several comments about Gradebook including: it is daunting, slow, and difficult to use, missing the F-‐ option, is complicated, users experienced numerous problems with it, needs more options, lacks flexibility, connections with eGrade are intermittent and some users prefer Class Record (an in-‐house software program, developed by Dr. Robert Pinto, Professor Emeritus, Department of Philosophy). Instructors/staff commented on the difficulties they faced in trying to facilitate discussions online using the Discussions and Forums tools, with complaints about load time, layout, notifications, marking posts as read, and using discussions with groups. There were also issues with the text editors in various tools, lack of flexibility in the email tool (i.e., the ability to email just one student, and for students to be able to email each other), navigation, differences in tools in terms of look and feel, and the amount of time it takes to learn the system and to do anything within it. There was a perception that some of the tools were less useful or less likely to be used than others, including the Calendar and Wiki, while others (including discussion-‐based tools, Gradebook,

NOTESNOTES ACCESSACCESSANNOUNCEMENTSANNOUNCEMENTS RESOURCESRESOURCESEMAILEMAIL INFORMATIONINFORMATION EASYEASY ABILITYABILITYCLASSESCLASSES LECTURELECTURE GRADESGRADES ONLINEONLINE POSTEDPOSTED POSTPOSTSYLLABUSSYLLABUS DISCUSSIONDISCUSSION SECTIONSECTION PLACEPLACE ASPECTASPECT COURSESCOURSESGRADEBOOKGRADEBOOK SLIDESSLIDES EASILYEASILY CHATCHAT LECTURESLECTURES MARKSMARKS COMMUNICATECOMMUNICATEFEEDBACKFEEDBACK MATERIALMATERIAL MATERIALSMATERIALS SITESITE POSTINGPOSTING TIMETIME BOARDSBOARDS CHECKCHECK HELPFULHELPFULIMPORTANTIMPORTANT UPDATESUPDATES COMMUNICATIONCOMMUNICATION MAILMAIL TATA ANNOUNCEMENTANNOUNCEMENT EMAILSEMAILS CONTACTCONTACT DATESDATESDOCUMENTSDOCUMENTS DOWNLOADDOWNLOAD LESSONSLESSONS RESOURCERESOURCE AREAAREA GREATGREAT NOTIFICATIONSNOTIFICATIONS ASSIGNMENTASSIGNMENT FILESFILES PAGEPAGE TOOLTOOLCALENDARCALENDAR FACTFACT LOCATIONLOCATION POWERPOINTSPOWERPOINTS QUESTIONSQUESTIONS VIEWVIEW WEBSITEWEBSITE ACCESSIBLEACCESSIBLE BOARDBOARD EASIEREASIER SYSTEMSYSTEMCONVENIENTCONVENIENT FEATUREFEATURE LEARNINGLEARNING TABTAB UPLOADUPLOAD CONTENTCONTENT FORUMSFORUMS POWERPOINTPOWERPOINT ACCESSINGACCESSING AVAILABILITYAVAILABILITY BOXBOX DATEDATEDIRECTLYDIRECTLY MESSAGESMESSAGES NAVIGATENAVIGATE ORGANIZEDORGANIZED POSTSPOSTS SENDSEND SUBMITSUBMIT


2012

34 | P a g e

Assignments, and Quizzes & Tests) could be useful if they were more user-‐friendly. There was also a strong perception in the qualitative data that there were numerous instability issues with functions like uploading or attaching files, files going missing or becoming unavailable, temporary unexplained “glitches,” and even access to the system.

Figure 19. Instructors/Staff: Worst or Least Useful CLEW Features STUDENTS Students provided a large volume of feedback on what they perceived as the worst or least useful aspects of CLEW. While there were very specific and detailed comments about individual tools in CLEW, their comments also frequently related to the way in which tools were used. For example, they overwhelmingly perceived that the calendar tool was the least useful aspect of CLEW but most qualified this by saying this perception was because it was not used at all or not used appropriately in their courses. Some students suggested the calendar should be mandatory in all CLEW sites and should contain items such as due dates for assignments, class times, University important dates, and holidays, as well as being customizable and able to be shared to their other calendars (e.g. Google, Outlook etc). There were many comments about CLEW not being used at all, used only to post resources, or used inconsistently from course to course (Figure 20). There was also a large number of students who were unhappy with tools being made available but not used (e.g., resources shown but no resources added). There was a strong perception that tools that facilitate interaction (Chat, Discussions, Forums) were not widely used and were not user friendly when they were used. Similar to comments from instructors, students who responded to this question felt that the system was slow, not intuitive, difficult to navigate, cumbersome to use, and took too many clicks to achieve results. There were also comments about having multiple options that are fairly similar (e.g. Discussions and Forums; Gradebook, Assignments, and Feedback) which they found confusing, and the My Workspace tab was not perceived as useful or as being used. Students frequently commented on the number of different systems they were forced to use because many instructors were not using CLEW or were only using CLEW for a portion of the learning activities


2012

35 | P a g e

and using multiple other systems for things such as access to content or quizzing. They felt that there should be better integration between myUWindsor (the student portal) and CLEW, and with the student email system. As with instructors, students frequently commented on the limitations of the email system in CLEW, indicating that they wanted to be able to contact anyone in the course directly. Many expressed frustration at being able to see who was online, but not able to interact with them. Students felt that some instructors over-‐used email and announcements (particularly when students were emailed every time a new resource was added), and that they would prefer to be able to subscribe to particular sections of CLEW for announcements about updates. Problems with Web browsers and operating system (OS) compatibility were indicated as a major source of frustration. Additionally there was a perception that CLEW was frequently “down” or under maintenance. Finally, students found it annoying that that course codes were used to identify courses on tabs rather than intuitive, customizable titles.

Figure 20. Students: Worst or Least Useful CLEW Features MOST IMPORTANT IMPROVEMENTS INSTRUCTORS/STAFF Instructors/staff provided useful feedback on improvements they would like to see included in CLEW when asked: What is the most important improvement you would like to see? The most frequent requests included: a cleaner more intuitive design/user interface; improved speed and uptime; improved assessment functions (including Gradebook) to make them easier and faster to use; and better handling of groups (Figure 21). The most frequent specific requests included: improved integration between eGrades and Gradebook; greater functionality of Gradebook in general; ability to create your own assessment quizzes/tests in a

CALENDARCALENDAR CHATCHAT DISCUSSIONDISCUSSIONPAGEPAGE POSTPOST INFORMATIONINFORMATION CLASSESCLASSES EMAILEMAILSITESITE RESOURCESRESOURCES COURSESCOURSES SYSTEMSYSTEM CALENDERCALENDERGRADEBOOKGRADEBOOK TABSTABS ANNOUNCEMENTSANNOUNCEMENTS ONLINEONLINE SECTIONSECTIONFEEDBACKFEEDBACK USELESSUSELESS WORKSPACEWORKSPACE TABTAB CONFUSINGCONFUSING LESSONSLESSONSPOSTEDPOSTED USERUSER FRIENDLYFRIENDLY NOTESNOTES HARDHARD PEOPLEPEOPLE PROFILEPROFILE ACCESSACCESS BOARDSBOARDSCLICKCLICK LAYOUTLAYOUT LISTLIST MAKEMAKE NAVIGATENAVIGATE WEBSITEWEBSITE EMAILSEMAILS SECTIONSSECTIONS SYLLABUSSYLLABUSDIFFICULTDIFFICULT MARKSMARKS TOPTOP WORKWORK BOARDBOARD FORUMFORUM GRADESGRADES INTERFACEINTERFACE SLOWSLOW UNIVERSITYUNIVERSITYLECTURELECTURE ORGANIZEDORGANIZED PROPERLYPROPERLY


2012

36 | P a g e

user-‐friendly way; improved speed/load time in all tools; easier navigation and cleaner layout; improved support for learning how to use CLEW; look and feel to be more like other familiar web-‐based applications; replacing the course code with course name in site tabs; and improved discussions tools (easier to use and navigate). There were also frequent requests for a more flexible email tool that allows the instructor to email a single student, and allows the students to contact each other, integration with a video/multimedia tool, and more customization for the individual user. There were also requests for a real-‐time synchronous communication tool, a range of course templates as examples/starting points for building a CLEW site, and more web 2.0 applications.

Figure 21. Instructors/Staff: Most Important Improvements in CLEW by Themes

ACCESSACCESS SUPPORTSUPPORTUSERUSER SYSTEMSYSTEM EMAILEMAILFRIENDLYFRIENDLY GRADEBOOKGRADEBOOKINFORMATIONINFORMATION EASIEREASIER TOOLTOOL EASEEASEINTERFACEINTERFACE SITESITE GRADESGRADES HELPFULHELPFUL IMPORTANTIMPORTANT TRAININGTRAININGUPLOADUPLOAD COMMENTCOMMENT EASYEASY GRADINGGRADING LEARNLEARN NICENICE RESOURCESRESOURCES SITESSITES SOFTWARESOFTWARE TEXTTEXTTOOLSTOOLS


2012

37 | P a g e

STUDENTS Students made many suggestions for improving CLEW and as with their perceptions of the worst or least useful features, many of these (such as making CLEW use compulsory in all courses, and mandating certain types of use) relate to teaching behaviours rather than aspects of the system itself. Nevertheless, they did make many suggestions for potential improvements to the system. The major themes of these requests include improving the navigation and usability of the interface, improving speed and uptime, integrating with other UWindsor systems (such as webmail, SIS, myUWindsor, etc.), and improving communication and collaboration functions (Figure 22). Students expressed frustration at the lack of consistency between tools, and between the tools used in different course sites. Some more specific suggestions for instructor use of the system included making it mandatory to post grades to Gradebook, use the calendar to indicate all important dates, choose only one discussion tool, and to use the communication tools consistently and judiciously. System changes that were most frequently requested included a cleaner design and consistent navigation between tools, better integration with other UWindsor systems (a one-‐stop-‐shop), better/more flexible text editing tools including a spell check and easier pasting from Word, improved Web browser and operating system compatibility, increased speed of the system, integration with web 2.0 tools and social media, cleaner layout/navigation, the ability to email classmates through CLEW, the use of course name instead of course code for site identification, the ability to sync the CLEW calendar with other calendars (e.g., Google, Outlook), and the ability to use the browser “back” button. Students would also like to be able to personalize their CLEW sites, interact or join classes in an online real-‐time virtual sense, a consistent look and feel from tool to tool, and a reduction in the number of clicks required to access the desired resources. A relatively large proportion of responses also requested that Gradebook be able to show the current grade standing of students and predicted grade based on current standing, as well as class average, median, and other statistics.


2012

38 | P a g e

Figure 22. Students: Most Important Improvements in CLEW by Themes WHAT DO WE NEED IN AN LMS? INSTRUCTORS/STAFF Instructors/staff were asked what, from their perspective, what was needed in a learning management system to make it effective in supporting teaching and learning (Appendix 1). There were three major themes in their responses: a system that allows easy communication and transmission of information, a user-‐friendly system that is easy to navigate and highly usable, and multiple levels of support for instructors to help them use the system (Figure 23). Specifically, instructors felt that an LMS should be primarily easy to use with an interface that is more like other familiar applications, and that it should seamlessly integrate with other applications. There was a feeling that the core applications should be highly robust and user-‐friendly, that the system should be easily extensible as needed, and that it should be very responsive (in terms of speed of access). Others felt that the LMS should be a place where resources could be stored, announcements made, and collaboration facilitated (including in real-‐time), and that it must be highly robust, reliable and have absolutely minimal downtime. There were a number of suggestions that the system needs to

EASIEREASIER SITESITE CLASSESCLASSESCOURSESCOURSES INFORMATIONINFORMATION PAGEPAGESYSTEMSYSTEM IMPORTANTIMPORTANT TABSTABS CHATCHATEMAILEMAIL RESOURCESRESOURCES ANNOUNCEMENTSANNOUNCEMENTSCALENDARCALENDAR POSTPOST USERUSER DISCUSSIONDISCUSSION GRADESGRADESCLICKCLICK ACCESSACCESS LAYOUTLAYOUT WEBSITEWEBSITE ONLINEONLINE POSTEDPOSTEDSECTIONSECTION NAVIGATENAVIGATE NOTESNOTES WORKWORK FRIENDLYFRIENDLY TABTAB CHECKCHECKDATESDATES OPTIONOPTION TOPTOP COMMUNICATECOMMUNICATE FASTERFASTER MARKSMARKS TIMESTIMESIMPROVEMENTSIMPROVEMENTS NUMBERNUMBER ORGANIZATIONORGANIZATION LESSONSLESSONS ORGANIZEDORGANIZED ABILITYABILITYDIFFICULTDIFFICULT EASYEASY FILESFILES GOODGOOD INTERFACEINTERFACE QUESTIONSQUESTIONS UPDATEUPDATE GREATGREAT LISTLISTLOGLOG MATERIALMATERIAL MESSAGESMESSAGES PEOPLEPEOPLE SIDESIDE BOXBOX CALENDERCALENDER LINKSLINKS LOTLOT MAKINGMAKINGMANDATORYMANDATORY NAVIGATIONNAVIGATION NICENICE SHOWSHOW SITESSITES SYLLABUSSYLLABUS ACCESSIBLEACCESSIBLE BARBAR BOARDBOARDCHANGECHANGE COMMUNICATIONCOMMUNICATION GRADEBOOKGRADEBOOK LIVELIVE OPTIONSOPTIONS POSTINGPOSTING RECEIVERECEIVE UPLOADUPLOADEASILYEASILY FEELFEEL FILEFILE HELPFULHELPFUL VIEWVIEW WEBMAILWEBMAIL CONSISTENCYCONSISTENCY CONTENTCONTENT DOWNLOADDOWNLOAD FINEFINEFUNCTIONFUNCTION GROUPGROUP HOMEHOME IMPROVEIMPROVE LEFTLEFT LINKLINK MESSAGEMESSAGE


2012

39 | P a g e

handle assessment of student learning and reporting of grades as an integrated and robust package. There were also a relatively large number of suggestions for a variety of synchronous and asynchronous communication tools to be included in any LMS, including video and audio in real time (and recorded), easy to use and intuitive threaded discussion forums, and text-‐based chat.

Figure 23. Instructors/Staff Perceptions of What is Needed in an LMS

STUDENTS The student perspective on the question – From your perspective, what do we need in a Learning Management System, such as CLEW, for it to be effective in supporting teaching and learning at the University of Windsor? – provided very similar key themes to those raised by the instructors, but with much more detailed suggestions. Students primarily believe an LMS should be easy to use, navigate, and user-‐friendly. They felt that the LMS should be a one-‐stop shop for communication, information and announcements, resources, assessment, grades, and structured lessons (Figure 24). Unlike instructors, they were not as concerned about support mechanisms, but indicated that good help documentation and online support is important. Students reiterated that the best LMS is not useful unless instructors use it appropriately and consistently. To this end, they suggested that templates and training were needed to make it easier for instructors to use the tools in the LMS in the best possible way to support learning. Communication and collaboration tools featured prominently in student comments on this question, with many suggesting these were the most important features to support learning. Students also suggested that integration with other systems such as email, calendars, and student portals was important, as was the ability to subscribe to the components they wanted to be updated on. Further, they suggested that the system should be fast, responsive, and take minimal clicks to get to all desired content. Cross-‐platform compatibility and extensibility, along with reliability and consistent uptime, were also discussed as critical elements to success. Mobile device access was indicated as an important emerging element that LMS designers should consider.

SYSTEMSYSTEM EASYEASY INFORMATIONINFORMATIONUSERUSER ACCESSACCESS EASEEASE LEARNINGLEARNING SUPPORTSUPPORTFRIENDLYFRIENDLY TEACHINGTEACHING FACULTYFACULTY TRAININGTRAINING COMMUNICATIONCOMMUNICATION COURSESCOURSES STAFFSTAFF


2012

40 | P a g e

Figure 24. Student Perceptions of What is Needed in an LMS WHAT WOULD MAKE CLEW EASIER FOR YOU TO USE? INSTRUCTORS/STAFF Instructors/staff felt that the most important thing that would make CLEW easier to use was more training and support in different formats such as online (live) help or training, training after hours, video tutorials, and improved online and inline help (Figure 25a). Gradebook also featured prominently with suggestions that streamlining grading and Gradebook would make the system easier. There were several comments about being able to create assessment tasks, including quizzes and different types of online exams, in CLEW without having to contact an administrator. Instructors also felt that it was necessary to redesign the interface to make it more intuitive, and make the navigation simpler. There was a sense that the tools in CLEW could be useful, but they were often clunky, slow, and not intuitive. Most instructors who responded felt that there were far too many steps to achieving even simple changes in the site, and that it, therefore, took a long time to set up a site. The discussions tools were perceived as being potentially useful but overly complicated and not user-‐friendly.

Figure 25a. Instructors/Staff: What Would Make CLEW Easier to Use?

EASYEASY INFORMATIONINFORMATION SYSTEMSYSTEM RESOURCESRESOURCES EFFECTIVEEFFECTIVECOURSESCOURSES CHATCHAT POSTPOST SITESITE ACCESSACCESS LEARNINGLEARNING CLASSESCLASSES COMMUNICATIONCOMMUNICATIONNOTESNOTES DISCUSSIONDISCUSSION EASIEREASIER GRADESGRADES EMAILEMAIL ANNOUNCEMENTSANNOUNCEMENTS GOODGOOD ONLINEONLINE USERUSERFRIENDLYFRIENDLY ORGANIZEDORGANIZED COMMUNICATECOMMUNICATE NAVIGATENAVIGATE SYLLABUSSYLLABUS POSTEDPOSTED PLACEPLACE HELPFULHELPFUL MATERIALMATERIAL GREATGREATLECTURELECTURE WEBSITEWEBSITE IMPORTANTIMPORTANT CALENDARCALENDAR LESSONSLESSONS MARKSMARKS CONTACTCONTACT PAGEPAGE UPLOADUPLOAD EASILYEASILY QUESTIONSQUESTIONS TOOLTOOL FEELFEELINTERFACEINTERFACE BOARDSBOARDS GRADEBOOKGRADEBOOK SECTIONSECTION DATESDATES TABSTABS LECTURESLECTURES MANAGEMENTMANAGEMENT SIMPLESIMPLE WORKWORK ASSIGNMENTASSIGNMENT FILESFILES FINEFINE PROPERLYPROPERLY SETSET ACCESSIBLEACCESSIBLEBOARDBOARD INTERACTIVEINTERACTIVE LINKSLINKS OPENOPEN OPTIONOPTION TIMESTIMES MATERIALSMATERIALS OPTIONSOPTIONS ORDERORDER POSTINGPOSTING UNIVERSITYUNIVERSITY UPDATESUPDATES LAYOUTLAYOUT MAKESMAKES TEACHINGTEACHING CLEARCLEAR DATEDATEDIFFICULTDIFFICULT DOCUMENTSDOCUMENTS FEEDBACKFEEDBACK LISTLIST NAVIGATIONNAVIGATION SLIDESSLIDES WORKSWORKS ADDADD CONSISTENCYCONSISTENCY DISCUSSIONSDISCUSSIONS EFFECTIVELYEFFECTIVELY FOLDERSFOLDERS PROGRAMPROGRAM BENEFICIALBENEFICIAL CONFUSINGCONFUSINGFORUMFORUM MAILMAIL MANDATORYMANDATORY ORGANIZATIONORGANIZATION SITESSITES UPDATEUPDATE VIEWVIEW AREAAREA EFFICIENTEFFICIENT FORUMSFORUMS INTERNETINTERNET REQUIREDREQUIRED SECTIONSSECTIONS SENDSEND SHOWSHOW SPECIFICSPECIFIC UNDERSTANDUNDERSTAND CHECKCHECK CLASSROOMCLASSROOMCONTENTCONTENT CURRENTCURRENT EASEEASE EXAMEXAM PEOPLEPEOPLE PRETTYPRETTY QUICKLYQUICKLY SUPPORTSUPPORT WEBSITESWEBSITES


2012

41 | P a g e

STUDENTS Similar to instructor comments, student comments related mostly to interface design. The most important thing from the student perspective that would make CLEW easier to use was an easier to use layout and navigation (Figure 25b). Those who responded felt the design was too busy, it was too difficult to get access to the resources needed, and it takes far too many clicks to access areas of the site. There was also discussion of changing the way in which the tabs are set up such as defaulting to more tabs available across the screen and having access to other key resources (such as the library, myUWindsor, etc.) as tabs. Students would also like to see video tutorials of key activities for students in CLEW. Mobile accessibility and increased speed were also discussed as options that would make it easier to use, as was enforcing greater consistency of use by instructors. They also suggested that there should be only one tool for each function, e.g., Discussions or Forums, Feedback or Gradebook, etc.

Figure 25b. Students: What Would Make CLEW Easier to Use?

EASIEREASIER TABSTABS COURSESCOURSESLAYOUTLAYOUT RESOURCESRESOURCES INTERFACEINTERFACE NAVIGATIONNAVIGATION PAGEPAGECLASSESCLASSES NAMESNAMES


2012

42 | P a g e

Key Quantitative Statistics EASE OF USE Both instructors and students were asked the question: How easy do you think CLEW is for students to use? The instructors also had the option of providing “No Opinion” (21% of responses), and were asked an additional question about feedback they had received from students on their use of CLEW. Several instructors indicated in their comments that it would be best to ask the students this question directly (which was also done – see Figure 26 for the comparison). The purpose of the question was to gauge whether there was a disconnect between the perceptions of instructors about how easy CLEW is to use and how students felt about it, and also recognizing that most support for students comes from instructors and GAs, it was important to identify the feedback instructors receive from their students on using CLEW. The majority of students (76%) indicated that overall CLEW was either “Easy” or “Very Easy” to use, while the instructors/staff who answered predicted that only 52% of students would find it “Easy” or “Very Easy” to use (Figure 26).

Figure 26: Instructor and Student Responses: How easy do you think CLEW is for students to use?11

11 Note: “No opinion” was omitted in the student version of the survey.

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Very easy Easy Neither difficult or

easy

Difficult Very difficult No Opinion

Percen

tage

Ease of use raengs

Instructor/staff Students


2012

43 | P a g e

When asked – How easy do you think CLEW is to use? – instructors/staff found CLEW more difficult to use than students, with 47% rating it as “Difficult” or “Moderate” as compared to only 23% of students finding it “Difficult” or “Moderate” (Figure 27). Likely, this is the result of more complex tasks needed to set up and manage courses, and is consistent with the practice of providing more support to instructors/staff (workshops, online, and customized assistance) than to students.

Figure 27. Instructor/Staff Perceptions on CLEW Usability for Instructors and Staff The additional comments from instructors about the ease of using CLEW focused on inconsistencies between tools in the way they look and operate (a function of the system being open-‐sourced and developed by different parts of the community with different goals and purposes in mind), the amount of time it takes to achieve simple tasks (e.g., the number of clicks taken), and the difficulty of trying to work out how to complete a new task (comments suggested that the interface and interaction was not always intuitive). Students generally felt that CLEW was easier for them to use than for their instructors, and to use it well, computer proficiency was necessary. They also noted the variance between tools and their interfaces within CLEW and indicated that this made it more difficult to use. In response to the question about feedback instructors had received from students about using CLEW, some indicated that they rarely heard feedback unless there was a problem, and that Mac and other personal computer users reported compatibility issues, including that the system was relatively slow to load and use. There was a perception that most students wanted instructors to use CLEW more, and to

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50% Pe

rcen

tage

Instructor / staff raengs ease of use


2012

44 | P a g e

use it more consistently, and that older students and graduate students found it more challenging to use. There were complaints about CLEW not supporting collaborative work or group work. Instructors indicated that students were appreciative of being able to study at their own pace or access learning when their schedule permitted and that they generally found it useful. There was a strong perception that students just take CLEW for granted and demand that it be used in every course. HOW DOES CLEW COMPARE TO OTHER LMS’ RESPONDENTS HAVE USED? INSTRUCTORS/STAFF The most common alternative LMS that instructors had used were Blackboard Learn (45%) and WebCT (31%), followed by ViCKi (22%), which was the University of Windsor’s in-‐house LMS prior to CLEW and included under the “Other” category, Moodle (10%), Desire2Learn (10%), and Angel (8%). See Figure 28 for a comparison. Several instructors, however, found CLEW easier to use than the previous institutional LMS (ViCKi) or older versions of WebCT and Moodle. Figure 29 indicates that 60% of instructors found it similar (30%), better, or much better than other LMS’ they have used. The majority of instructors who provided comments suggested the alternative LMS (mainly Blackboard) was more intuitive, easier to learn, had a better graphical user interface, was able to do group-‐work better, copy documents easier, and was simpler to use. STUDENTS Students did not provide many comments on other LMS’ they have used although the quantitative data (Figure 28) indicated that many had used other systems. From the comments received, most indicated it was easier to use than ViCKi, Blackboard, or Angel. The students rated CLEW higher than instructors/staff compared to other systems, and 79% indicated CLEW was “Moderate” to “Very Easy” compared to the other LMS’ (Figure 29).


2012

45 | P a g e

Figure 28. Comparison of Other LMS Use (Instructors and Students)

Figure 29. User Rating of CLEW Compared to Other Learning Management Systems

0% 10% 20% 30% 40% 50%

Other, please specify:

Blackboard Learn

Desire2Learn

Moodle

WebCT

Angel

Percentage

Other LMS Students

Instructor / Staff

0% 10% 20% 30% 40%

Other, please specify:

Haven't used CLEW so can't compare/no opinion

Much worse/very difficult

Worse/difficult

Similar/moderate

Bewer/easy

Much bewer/very easy

Percentage

Raen

g

Students

Instructor / Staff


2012

46 | P a g e

Quantitative Statistical Analysis INSTRUCTOR/STAFF DATA CORRELATIONAL DATA Correlational analysis was performed to search for patterns between all the factors measured. Only statistically significant correlations are reported here. In summary:

• There was a negative correlation between the number of courses taught using CLEW and the instructor’s belief that it could be used to support student interaction, encourage group work, or peer review. This indicates that as instructors become more familiar with CLEW and its tools, they perceive it less favourably for supporting interactive learning or tasks that require communication. Those who had taught more courses in CLEW also tended to rate other LMS’ higher.

• Those who had taught more courses in CLEW tend to use announcements more frequently. In addition, those who frequently use Announcements or Forums are more likely to use other tools such as Assignments, Discussions, and Resources. Those who use the Chat tool are also significantly more likely to use more tools that go beyond broadcasting information, such as Polls, Wiki, Feedback, Forums, Lessons, and Email Instructor.

• Those who believed CLEW could be used to support student-‐to-‐student interactions believed it could also be used to support student-‐to-‐professor interactions, group work, peer review, self-‐reflection, and assessing student work. This indicates that some instructors believe the system is able to support some of these higher-‐level functions. Those who use Discussions and Forums believe CLEW can be used to support interactivity.

• Those who have used other LMS’ believe that they are easier for students to use, and support group work, peer review, self-‐reflection, and assessment of student work better.

• Those who have more years as an instructor tended to believe that CLEW did not support student-‐to-‐student interaction.

LIKERT-‐SCALE QUESTION DATA SUMMARY Instructors were asked to rate a range of items related to CLEW on a Likert scale. Mean responses to these questions provide some interesting patterns. In summary:

• Instructors in general rated CLEW relatively poorly for supporting group work (mean = 2.99/6), peer review (2.33), self-‐reflection (2.63), and student-‐to-‐student interaction (3.25). Instructors generally believed CLEW would be easy for students to use.

• Overall, support was rated very highly (with qualitative comments related to this question indicating excellent support from the LMS Administrator and the Learning Technologies Educational Consultant).

• As part of the support subscale, the system was rated highly for stability and reliability, but lower for usability and support for pedagogical issues.

T-‐TESTS Univariate t-‐Test comparisons also reveal some patterns of note. The instructor’s ease of use perceptions were used as a baseline indicator for comparison across factors. In summary:


2012

47 | P a g e

• Instructors who had not used another LMS rated CLEW higher than those who had used other

LMS; although this was not a significant difference. Breaking this down to individual LMS, those who had used Blackboard or Moodle rated CLEW significantly lower, while those who had used ViCKi or others rated CLEW higher for ease of use (although not significantly).

• There was no significant difference (NSD) between instructors’ rating of ease of use and whether they would recommend it to their colleagues (almost all said they would recommend it to other instructors).

• There was NSD between ease of use perception and the faculty with which the instructor was based. Graduate Studies, Human Kinetics, Business, GLIER, Engineering, and Law all rated CLEW’s ease of use above the mean (not significantly), while Education, Nursing, and Science instructors all rated it below the mean (also not significantly).

• Male and female instructors found it equally as easy to use. STUDENT DATA CORRELATIONAL DATA Correlational analysis was performed to search for patterns between all the factors measured. In the student data there were not as many strong patterns as seen in the instructor/staff data. Only statistically significant correlations are reported here. In summary:

• There was a positive correlation between how easy students perceived the system to be and how frequently they used it, indicating that those who spend more time on CLEW feel it is easier to use. This may be a function of familiarity, or it may indicate that those who think it is fairly easy to use are more likely to use it.

• There was a negative correlation between perceived computer expertise and how often students visit the site, with students who believe they have less expertise with computers indicating they visit CLEW sites more frequently.

SUMMARY OF LIKERT-‐SCALE QUESTION DATA Students rated a range of items related to CLEW on a Likert scale. Mean responses to these questions provide some interesting patterns. In summary:

• Students considered group work in CLEW problematic. Most said that they had not used CLEW for group work. Those who used tools other than CLEW to facilitate group work rated CLEW higher than those who tried to use CLEW for group work, suggesting CLEW is better for broadcasting tools. See Appendix 3, Figure 31, for a quantitative view of which CLEW tools were used for group work.

• First-‐year students tend to find CLEW easier to use than those in subsequent years, students in professional programs, or post-‐graduate students (significantly harder).

T-‐TESTS Univariate t-‐Test comparisons also reveal some patterns of note. The students’ ease of use perceptions were used as a baseline indicator for comparison across factors. In summary:


2012

48 | P a g e

• Similar to instructors, students who had used another LMS (approximately one third of students) rated CLEW significantly more difficult to use than those who had not used another LMS.

• In contrast to instructors, of those who have used another LMS, for example, Blackboard, did not rate CLEW significantly harder to use, but those who had used Moodle, WebCT, and other LMS rated CLEW significantly more difficult to use.

• Also in contrast to instructors, students from Education, Graduate Studies, Inter-‐faculty programs and Law rated CLEW significantly more difficult to use than their colleagues from other faculties.

• Professional programs and post-‐graduate programs use CLEW significantly less than their undergraduate counterparts.

MODELING AND PREDICTING EASE OF USE Considering responses to the ‘Ease of use’ perception as a proxy for usability, regression modeling was conducted to determine whether it was possible to predict which factors were most significant in contributing to this perception. The most important factors having a positive impact on perception of ease of use include: how often students visit the site, whether the Syllabus tool is used, whether group work for students has been facilitated in CLEW, and whether Announcements were used. Factors having a strong negative impact include: using the Wiki tool and if students have used another LMS. At the next level, using Gradebook, Resources, and Calendar can all have a positive impact, as can using CLEW in most of a student’s courses. LMS STRENGTHS AND WEAKNESSES The topic of the effectiveness of a learning management system has been discussed before. Lanny Arvin (2009), a former CIO and Associate Dean for eLearning in the College of Business at Illinois argues:

[T]he LMS serves as an affirming technology of traditional teaching. The instructor doesn’t challenge the LMS very much, and, in turn, the LMS doesn’t challenge the instructor. The student gets the convenience benefit from electronic distribution of documents (and grades) but little more.12

Jon Mott, Assistant to the Academic VP – Academic Technology Brigham Young University has identified the following strengths and limitations of a Learning Management System (Mott, 2010).13 Table 12. Generalized LMS Strengths and Weaknesses (Mott, 2010) LMS Strengths LMS Weaknesses

Simple, consistent, and structured [The way it is being used is] time-‐bound (courses

12 Lanny Arvan, "Dis-‐Integrating the LMS," EDUCAUSE Quarterly, vol. 32, no. 2 (April-‐June 2009). 13 Jonathan Mott, Envisioning the Post-‐LMS Era: The Open Learning Network, EDUCAUSE Quarterly Magazine, vol. 33, no. 1, 2010


2012

49 | P a g e

disappear at the end of the semester) Integration with student information systems (SIS), with student rosters automatically populated in courses

Teacher, rather than student, centred

Private and secure Courses walled off from each other and from the wider web, negating the potential of the network effect

Simple and inexpensive to train and support (compared to supporting multiple tools)

Limited opportunities for students to “own” and manage their learning experiences within and across courses

Tight tool integration such as quiz scores populated in Gradebook14

Rigid, non-‐modular tools

Supports sophisticated content structuring (sequencing, branching, adaptive release)15

Interoperability challenges and difficulties

The 2011 CLEW survey shows similar patterns at the University of Windsor. Major feature requests received by the CLEW Team and LMS Advisory Committee include: learning outcome tracking and analytics, adaptive and integrated quizzing tools, audience response systems and Scantron exam score integration with CLEW, archiving of courses, course access beyond the current semester for students, and ePortfolio options. These survey results and feature requests challenge the institution to think about the next phase of development, how we might address the maturation of the LMS, and the demand for more sophisticated functionality.

14 This is not available in CLEW yet, but does apply to Assignments, Forums, and Discussions. 15 This is not available in CLEW yet.


2012

50 | P a g e

Key Themes Tables 13 and 14 summarize the key qualitative themes derived from the open-‐ended feedback. Table 13. Instructors/Staff: Likes and Dislikes

Faculty & Staff

Like Dislike

The majority of instructors (92%) indicated they would recommend using CLEW to their colleagues

The functionality of grading and assessment tools (in CLEW) and Gradebook is considered daunting, slow and difficult to use

87% indicated that continuing to learn (about CLEW tools) was beneficial to them

The system speed (load time, access, changes), perceived instability issues, Web browser and operating system incompatibility issues

CLEW is a good place to store lecture notes and other course resources

Calendar and Wiki are less useful and not as likely to be used as other tools

CLEW makes it easy to communicate to all using Announcements, and is useful for grading and managing assignments

User interface is “clunky” and has the following issues: • Cluttered appearance in some tools (Lessons,

Resources, Discussions/Forums) • Navigation is too complex • Too many clicks to perform functions • Different look and feel between some tools

and is inflexible and not customizable • Text editor issues (pasting) and it has no spell

check Those who accessed support found it helpful Difficulties facilitating discussions using current tools

which have experienced load time issues, layout challenges, notification issues, and it is difficult to facilitate group work

Those who use the basic features find it works well The email tool is inflexible and unable to address individual students

Improving stability, reliability and support was an area of focus for 2007-‐2009 and the survey results note this by faculty / staff

• Want quizzes and exam-‐like assessment that instructors can initiate

Site Stats is useful to track who has used specific parts of site

Education, Graduate Studies, Inter-‐faculty programs and Law rate CLEW less favourably than other faculties


2012

51 | P a g e

Table 14. Students: Likes and Dislikes

Students Like Dislike The majority of students (76%) indicated

that CLEW was either “Easy” or “Very Easy” to use

Inconsistency. Students would like faculty to: use the tools as they were intended; use CLEW more; use CLEW consistently, disable tools that are not being used in a site, and not over use email notification and Announcements

Several students mentioned that they had few problems, and it basically works very well

Having multiple systems to use -‐ (some faculty use more than one system / website for a course) -‐ prefer one stop shop

Announcements Incompatibility issues with Web browsers and operating systems

Online access to Syllabus and grades was helpful

Downtime (some mentioned scheduled maintenance times as issue) and slow system speed

Easy access to class resources Calendar not being used or used inappropriately Majority of students visit CLEW daily. 4th

year and above spend less time in CLEW Navigation, as some find it difficult

Undergraduates -‐ 89% of 1st year students report at least one course uses CLEW

Forums and Discussion: students want intuitive interface, improved notification system, threading (ability to reply to a reply not just inline – Discussions only) and an easier way to tell read/unread status

By 4th year, 72% say more than 10 of their courses have used CLEW

Not being able to send emails to each other or the ability to interact / chat online with those online at the same time

Those who have used some other learning management systems are more critical of CLEW

Group work using CLEW tools; Students in classes that used 'something else' for group work are much more positive about their group work experience

Students in professional programs, Masters and PhDs find CLEW significantly harder to use than other groups and 15% of Masters and 22% of PhDs report not using CLEW at all


2012

52 | P a g e

Table 15 summarizes the instructor, staff, and student suggestions about what they believe is the most important and necessary CLEW improvement. Table 15. Suggestions for the Most Important CLEW Improvement

WOULD LIKE CLEW TO . . .

Instructors / Staff

Be easier and faster to learn, and have intuitive and customizable course tab names Have improved overall loading time Have a more intuitive interface, better design and look like other familiar web applications Be more integrated with Web 2.0 services (such as social networking or collaborative sites) Have improved Forums / Discussions interface and load time Have improved assessment tracking in Gradebook and more reliable eGrade integration Be cross-‐platform and cross-‐browser compatible Have a more flexible email tool Provide better handling of groups, and a virtual classroom tool for live discussions Have an instructor initiated quizzing tool

Students

Be more standardized between courses and instructor approaches Be mandatory for faculty to use Have less redundancy (there is currently duplication within some tools i.e. Discussions/ Forums and Gradebook / Feedback) and better integration with campus systems such as myUWindsor portal and Web Mail Have intuitive, customized tab names for courses and personalized course sites Have a better text editing tool that has spell check and can make pasting content in easier Have better Gradebook interface features and analytics (with students’ rank within the class, individual assessment average, etc.) Have an improved design/navigation and have instructors avoid using too many links in left hand menu Be faster (e.g. load time, refresh time) and experience less downtime Have a live chat help source for CLEW Integrate better with Web 2.0 features (text and chat), Google tools and integrate other calendars into CLEW Mandatory for faculty to post grades and add important dates into the Calendar Have an interface that looks like other familiar web applications

Recommendations A number of recommendations arise from the results of the survey that relate broadly to technical issues, design, and functionality of the system. Further recommendations relate to systemic changes in the management of CLEW and eLearning at the University. SYSTEM A number of issues were raised in the survey that require the intervention and development of the CLEW team. Some of the issues raised have already been addressed, others are in testing or development at present (Table 16). Many of the remainder can be addressed through in-‐house development, but some rely on the Sakai community, or are not possible in the current system. Table 16. System Recommendations Recommendation Timeframe

(Long, Medium or Short term, Complete)

Responsibility Comments

Improve load time for tools (Forums, Discussions, Resources).

C ITS Investigate cause of these issues – network, server space, software design. This was reviewed by the Applications Developer in August, and improvements to load time have been made.

Improve upload time for attachments (Resources, Drop Box, Lessons).

M ITS Investigate cause of these issues – network, server space, software design.

Improve integration with other University systems such as email, myUWindsor portal, campus calendar, SIS, and Leddy Library resources.

L ITS Develop a focus group to determine where improvements should be targeted.

Integrate CLEW with synchronous online collaborative tool (Blackboard Collaborate).

S ITS/CTL Testing integration.

Explore expanding browser/system compatibility.

M ITS Browser compatibility is part of the fundamental design of the system and it may not be possible to alter this. Contributing to the development of future versions that are compatible with more browsers may be the only option.

Create a mobile application to provide access to significant site information including grades, announcements, syllabus, discussion postings, lessons, resources, and other tools .

C ITS Completed and in production.

Provide more integration with Web 2.0 tools.

L ITS Sakai does not currently integrate with most web 2.0 tools.

Enable email between individuals within M ITS Investigate re-‐writing existing email tool


2012

54 | P a g e

a site. and potentially back port newer email tool from S2.9, which does provide this functionality.

Provide secure chat with notification features that can synchronously connect individuals within a site.

L ITS/CTL There is currently no plan in the Sakai community to develop the chat tool further. S2.9’s chat tool does have some new features (such as the ability to see students across courses and chat with them), but it is currently unstable and not being developed actively.

Provide course tabs that can be customizable.

L ITS/CTL Future iterations of Sakai may have this functionality, but this is not possible in the current version.

Develop tool “Wizards” to help instructors with more complicated tasks, such as setting up Gradebook and setting up Forums with Groups.

M CTL Better help for complicated tasks is currently under development.

Integrate with other teaching and learning tools such as clickers, video streaming, virtual classrooms, and Scantron results.

M ITS/CTL Local customizations can be developed in-‐house – priorities will be determined through faculty consultations and LMS Advisory.

Enable student access to their work beyond the current semester.

S ITS/CTL This is a matter of policy and storage space. It is strongly recommended the current policy of removing access to courses at the end of term be reviewed as soon as possible.

Improve consistency across tools for common functions such as “Hide and Show” features, editing, and saving.

M/L ITS/CTL This is a long-‐term goal of the Sakai community but not currently possible as there are no standards against which the OS tools are being built or a way to vet them other than for functionality. Some customization may be possible in some tools to improve consistency.

Improve handling of group activities such as group assignment, group assessment tasks, group discussions, ability to grade individually and as part of a group in Gradebook and assignments, and improve workflow for managing assignment of students to groups.

L ITS/CTL The current Sakai functionality is a barrier to these changes. It may not be possible to address any of these issues in the current or future versions of the CLE because it would require a complete redesign of the underlying structure of the LMS.

Exploration of a quizzing tool that instructors can initiate and manage all types of assessments or surveys through an intuitive interface, with Gradebook integration, adaptive release, analytics (including item/assessment analysis) and contain intuitive embedded help.

M ITS/CTL Investigate of options for quizzing/assessment tools that provide the requested functionality is underway. However, the leading Sakai assessment/quizzing tools are highly unstable and essentially unusable at present.

Improve the text editing tool to include functionality such as pasting from word processing programs, spell check, image

S/M ITS Pursue improved versions available in the Sakai community or other open source options. Some of the challenges posed by


2012

55 | P a g e

uploading, etc. the current text editor in CLEW will likely be addressed in the upgrade to S2.9.

Provide synchronous online collaboration tool for virtual classrooms.

C ITS/CTL BlackBoard Collaborate was piloted and deployed to campus Winter 2012.

USABILITY IMPROVEMENTS There is a need for a review of the usability of CLEW in light of much of the feedback from the survey that relates to usability issues users perceive (see Table 17). It is possible that many of these issues relate to the fundamental design of the system and will not be possible to change in the current version, but this feedback can be fed into the Sakai community for future iterations of the system. Some are also related to the way in which individual instructors choose to use the system and better support and education may help in this area. Table 17. Usability Improvement Recommendations Recommendation Timeframe



Encourage instructors to use CLEW in consistent ways, such as providing site templates for commonly used course approaches (i.e. highly interactive, integrated assessments, basic course).

M CTL Usability and the student experience is impacted by inconsistent use of tools. A multi-‐pronged approach is needed to improve education and support for instructors designing course activities in CLEW.

Review tool names to connect explicitly with functionality (verbs) vs. nouns.

S ITS/CTL It is possible to change the names of tools to better reflect their functions.

Provide recommendations on redesigning user interface, navigation, and ways to reduce number of clicks to accomplish common tasks, such as the browser back button.

M CTL/ITS/ Usability Review team

Many issues identified are as a result of the fundamental design of the system and cannot be changed.

Develop a way for users to subscribe to updates using mobile and other devices (e.g. announcements, grade postings and site updates)

C ITS The UWindsor mobile app continues to evolve and the usability of the features related to CLEW should be reviewed periodically. The ability to subscribe to updates from CLEW in various formats should be investigated further.

Move to single threaded discussion tool. M ITS/CTL Upgrade may resolve this concern. SUPPORT Support was generally well-‐received by the survey participants, however there were some areas in which support could be improved for all users (Table 18). Table 18: Support Recommendations


2012

56 | P a g e

Recommendation Timeframe (Long, Medium or Short term, Complete)


Offer training workshops virtually (online) during non-‐office hours.

S CTL Some sessions are currently delivered online, but the majority are face-‐to-‐face only. The CLEW support team should provide more offerings online or in flexible mode, which will require a reduction in the number of face-‐to-‐face offerings. As Blackboard Collaborate has become available as of Winter 2012, this is now an option.

Provide more training for GA/TAs as they appear to be the second most frequent source of support sought out by students who accessed any support.

S CTL GAs and TAs currently have access to all training offerings and specific sessions for these groups have been introduced to the GATAcademy for GAs and TAs. The CLEW support team works with departments and faculties who request specific training for their GAs/TAs but this service may not be widely known and could be better advertised.

Have a Help Desk presence at CLEW Team and LMS Advisory Committee meetings to keep up to date with current issues and provide feedback to CLEW Team about issues they experience since they are common first point of access.

C ITS A member of the Helpdesk staff is now present at these meetings.

Develop or link to videos that provide instruction on common key activities.

M CTL Development of videos can be labor-‐intensive but highly effective. More key functions need to have video tutorials as part of several earlier recommendations. Captioning would need to be embedded into any new training videos as per AODA guidelines.

Develop and provide access to “exemplar” sites from existing effective sites to encourage consistency and model best practices for instructors, staff, and GA/TAs.

M CTL/ITS Developing and collecting exemplar sites is relatively easy, but there is currently no easy way to provide access to those sites at a broader scale. CTL and ITS are in discussions to try and find a solution to this issue.

Develop a plan to understand use of existing support and improve uptake of all existing support by all stakeholders.

M CTL/ITS/LMS-‐related committees

Survey results indicate that those who access the existing supports are generally very satisfied with the support they receive. It is not possible to tell from the current results why users are not utilizing the existing support (e.g. unaware of it, can’t find it, don’t think it will help, or don’t have any issues) and understanding this is important to allow informed.


2012

57 | P a g e

decisions about where to expend energy in supporting users

Undertake future development planning for pedagogical support.

M/L CTL/ITS/VP-‐T&L/ O.L. Director

The results indicate that pedagogical support for instructors is rated lower than other supports. As instructors become more familiar and comfortable with the potential of online learning, they will outgrow the current infrastructure (which is already happening) and more sophisticated uses of the LMS will be required. To take this step however, pedagogical support and support for the design and development of appropriate learning environments is critical. The current level of resourcing in this area (human and technological) does not allow for future expansion and consideration needs to be given to how this mismatch between needs and resources might be managed in the longer term as the University community develops.

COMMUNICATIONS Improvements to the communication strategy for CLEW may help to improve campus engagement with the system and buy-‐in for major initiatives. New initiatives, upgrades, major feature requests and system issues are not currently communicated to the campus community as efficiently and effectively as possible (Table 19). Table 19: Communications Recommendations Recommendation Timeframe


Responsibility

Comments

Develop a plan to improve engagement of representatives on LMS Advisory Committee and ensure that discussions within the Committee are transmitted effectively to the University community.

S-‐M LMS Advisory/ LMS Steering

Major initiatives, system deficiencies, and upgrades are discussed with the LMS Advisory Committee and representatives on that committee should disseminate that information to their constituents. There has been generally poor attendance by a number of representatives on the LMS Advisory Committee.

Review role and mandate of the formal structures for CLEW (i.e. LMS Advisory) to create a more frequent and streamlined communication and review process for feature requests, similar to an expedited review committee.

M LMS Advisory/LMS Steering

The community would need to be consulted during the prioritization of feature requests. For example, this process could be similar to the expedited REB process.


2012

58 | P a g e

Develop a plan to disseminate information about known issues and resulting solutions to the campus community. The plan may include:

-‐ Technical support team and Helpdesk should make more information available via the CLEW webpage regarding known issues

-‐ Email notifications of unplanned outages should go to all users

-‐ Develop a process for dealing with feature requests and a means of informing the campus community of progress on these feature requests and explore a means for users to receive updates on requests

S-‐M CTL/ITS/LMS Advisory

There are a number of initiatives that would be possible to better engage with the community and inform them of what is happening with the LMS, although it should be noted that information is widely disseminated already through a range of on-‐campus channels.

The Learning Technologies Educational Consultant should actively disseminate information on major initiatives, upgrades, and known issues as part of the ongoing training sessions.

S CTL This is ongoing through workshops, campus events, the CLEW Help Wiki and Insight newsletter.

FORMAL PROCESSES FOR REVIEWING FEATURE REQUESTS GROUP There is a need for the development of a formal process for submitting, receiving, reviewing, prioritizing, and implementing feature requests and suggestions from the UWindsor community regarding the LMS. This could be achieved by LMS Steering determining the most appropriate team to undertake this review process re-‐advertising existing current web-‐based forms for making suggestions or indicating problems), (include chair of LMS Advisory access to the email communication chain) which would then form the agenda and send the suggestions to the expedited review team for processing and recommendations. The process would also need to include indications of how decisions and ongoing work are fed back to the Windsor community. Some example suggestions from the community that require in-‐depth review include:

• Ensuring better tool integration, ease of use, and tools that are more efficient when performing repetitive tasks.

• Reviewing redundancy in available tools (i.e. offer only one asynchronous discussion tool). • Grading function improvements and options such as adding bonus marks, dropping lowest

scores, provide more student analytics such as running average (per student and by class), student ability to see comparison of their mark to class average, ability to handle assessments other than points/percentages, pass/fail, automatic grade curving feature, predictive modeling, grading group work with batch operations, more intuitive self-‐help, identification of students at risk and when students have reviewed their grades.

• Assignments improvements to address uploading/downloading issues, performance and functionality, improvements for batch operations, group segmentation and grade recording


2012

59 | P a g e

• Address major request to track achievement of “Learning Outcomes” at multiple levels (course, major, program, degree etc.), which is not currently possible with CLEW, integration with other systems such as Scantron results, and clicker scores.

• Review policy for archiving courses, student access beyond course end dates. • Selection and implementation of an integrated ePortfolio system.

CONCLUSION Overall, the survey received a good response and there are a number of highlights to consider going forward: the majority of students (76%) indicated that CLEW was either “Easy” or “Very Easy” to use; the majority of instructors/staff (69%) rated CLEW either “Good” or “Excellent” in terms of stability, reliability and support for technical issues; 92% of instructors would recommend CLEW to a colleague to use and users were generally satisfied with the support they received. There is an overall preference for using and supporting one system on campus. Along with these highlights however, there were a number of issues and challenges that need to be faced to ensure that the University’s learning technology infrastructure is able to meet the needs of the university community. CLEW was seen by many as overly cumbersome, slow and lacking features of a contemporary LMS such as personalization, integration with social media, and effective assessment tools. It was also seen as not able to support group activities. To address the issues and deficits of the system however, requires a commitment to not just maintaining the system, but to improving it through a process of ongoing review and reevaluation. The LMS Review, which will be undertaken over the next calendar year conducted by an interdisciplinary team of stakeholders, will be the next step in the process for determining the future direction of the University of Windsor’s learning management system priorities. The University of Windsor faces significant pedagogical change in the next few years as more programs look to flexible modes of delivery in line with the changing face of post-‐secondary education in Ontario, along with pressure from Government to make such changes. A reliable, modern, efficient and usable learning management system is a critical piece of educational infrastructure that supports all learners. The outcomes of the 2011 CLEW Survey provide the University with a basis for understanding the state of the present LMS, the needs of its constituents, and possible future directions for the system.


2012

60 | P a g e

Appendices APPENDIX 1 -‐ CLEW SURVEY – INSTRUCTORS AND STAFF QUESTIONS I agree to participate in the survey.

Yes, I agree

No, I want to exit the survey.

If you do not wish to participate in this study, please close this window on your browser or navigate to another website now.

Do you use CLEW or another Learning Management System in your teaching?

Yes

No

No opinion

How did you use CLEW or another LMS in your teaching?

None

How many different courses have you used CLEW in? (Each offering is considered a new course e.g. 03-‐60-‐100 F10 and 03-‐60-‐100 W11 are different courses)

None

1-‐2

3-‐6

7-‐10

More than 10

What do you think is the best or most useful aspect of CLEW?

None

What do you think is the worst or least useful feature of CLEW?

None

Which CLEW tools have you used in your courses?

Don't know Never Rarely Sometimes Frequently Every course

Announcements


2012

61 | P a g e

Assignments

Calendar

Chat

Content Viewer

Discussions

Drop Box

Email Instructor

Feedback

Forums

Lessons

Glossary

Gradebook

News

Polls

Quizzes & Tests

Resources

Roster

Sections

Site Stats

Syllabus

Web Content

Wiki

Additional tool not in list above (please explain).

None

Please identify and rate your sources of support for CLEW.

Poor Mediocre Average Good Excellent Not used

Help Desk

CLEW Help wiki


2012

62 | P a g e

(http://www.uwindsor.ca/clew) **New support as of Fall 2011

CLEW email: [email protected]

LMS Administrator

Learning Technology Educational Consultant

“Your Comments” form on CLEW site

“Ask for help” form on CLEW site

Comments on the above rating(s).

None

How do you rate CLEW and CLEW Support on the following?

No opinion

Poor Mediocre Average Good Excellent

Stability (server up-‐time)

Reliability (does what it is supposed to do)

Usability (how easy user interface is to use)

Support for technical issues

Support for pedagogical (teaching issues)

What additional support for the use of CLEW would you like to see?

None

Is continuing to learn about CLEW tools beneficial to you?

Yes

No

Which CLEW Support workshops have you attended? Check all that apply:

CLEW Basics for Instructors and Assistants

CLEW Basics for Project Sites

Name Your Topic / Personal Consultation

Interactivity: Polls, Forums and Wiki


2012

63 | P a g e

Managing Student Work: Using Assignments and Drop Box

Gradebook Essentials

Lessons: The Step-‐by-‐Step Path to Student Learning

Gradebook, Feedback & Quizzes: Conducting and Recording Assessments

New Faculty Orientation

Nursing Orientation

Other, please specify: ______________________

Haven't taken any training

Which tool(s) or topic(s) of training (not listed above) are you interested in?

None

What format(s) do you prefer? (Select all that apply)

Do not want training

One-‐on-‐one

Small group (2-‐6 participants)

Mid-‐size group (7-‐15 participants)

Large group (15 or more)

Comments

None

To what degree do you find you can use CLEW to encourage the following;

Don't know Never Rarely Sometimes Frequently Always

Student-‐to-‐professor interaction

Student-‐student interaction

Group-‐work

Peer-‐review

Self-‐reflection

Assessment of student work

Comment on your experience


2012

64 | P a g e

None

How easy do you think CLEW is for instructors to use?

No opinion

Very difficult

Difficult

Moderate

Easy

Very easy

Comments

None

What would make CLEW easier for you to use?

None

Why not and what would make you want to use an LMS such as CLEW?

None

From your perspective, what do we need in a Learning Management System such as CLEW, for it to be effective in supporting teaching and learning at the University of Windsor?

None

What is the most important improvement you would like to see?

None

If you use CLEW or another learning management system, do you use it differently in different classes e.g. large, small, undergraduate, graduate? (Select all that apply).

Use it differently with large vs. small classes (please comment) ______________________

Use it differently with undergraduate vs. graduate (please comment) ______________________

Don't notice any difference

Don't know


2012

65 | P a g e

If Yes, in what ways do you use it differently?

None

How easy do you think CLEW is for students to use?

No opinion

Very difficult

Difficult

Neither difficult or easy

Easy

Very easy

What would make CLEW easier for students to use?

None

What feedback have you received from students on your use of CLEW or another LMS in your teaching?

None

Have you used other learning management systems?

Yes

No

Please check all that apply.

Blackboard Learn

Desire2Learn

Moodle

WebCT

Angel


How does CLEW compare to the other systems you’ve used?

Haven't used CLEW so can't compare

Much worse


2012

66 | P a g e

Worse

Similar

Better

Much better


Comments

None

Would you recommend that a colleague use CLEW in their teaching?

Yes

No

Please explain.

None

Have you used any non-‐CLEW collaborative technologies for your course or project sites?

Yes

No

Please describe how you have you used them?

My own website ______________________

Google Docs ______________________

Word Press ______________________

Other external tools or systems, please specify ______________________

DEMOGRAPHICS With which Faculty are you associated? Check all that apply.

Faculty of Arts & Social Sciences

Faculty of Education

Faculty of Graduate Studies

Faculty of Human Kinetics

Faculty of Nursing


2012

67 | P a g e

Odette School of Business

Faculty of Science

Inter-‐Faculty Programs

Great Lakes Institute for Environmental Research

Faculty of Engineering

Faculty of Law


Rate your overall expertise with computers relative to other instructors or staff at the University of Windsor.

Novice

Intermediate

Advanced

Prefer not to answer

How many years have you been an instructor / faculty member / staff member in higher education?

1 year or less

2-‐5 years

6-‐10 years

11-‐20 years

21-‐30 years

More than 30 years


What is your gender?

Male

Female



2012

68 | P a g e

APPENDIX 2 – CLEW SURVEY – STUDENT QUESTIONS Do you agree to participate?

Yes, I agree

No, I want to exit the survey.

If you do not wish to participate in this study, please close this window on your browser or navigate to another website now.

This semester, how often do you typically visit your CLEW site(s)?

None of my courses are using CLEW

A few times in a semester

A few times in a month

Once a week

A few times in a week

Daily (once or more)

How many of your courses have used CLEW? (e.g. 03-‐60-‐100-‐01 F10 and 01-‐01-‐150-‐02 W11 are two different courses)

Have never used CLEW

1-‐5 courses

6-‐10 courses

More than 10 courses

From your perspective, what do we need in a Learning Management System, such as CLEW, for it to be effective in supporting teaching and learning at the University of Windsor?

None

What is the most important improvement you would like to see?

None

If your instructors use CLEW or another learning management system, do you find they use it differently for different types of classes e.g. large, small, undergraduate, graduate? (Select all that apply).

Use it differently with large vs. small classes (please comment) ______________________


2012

69 | P a g e

Use it differently with undergraduate vs. graduate classes ______________________

Don’t notice any difference

Don’t know

What do you think is the best or most useful aspect of CLEW?

None

What do you think is the worst or least useful aspect of CLEW?

None

Which CLEW tools have you used in your courses?

Never Rarely Sometimes Frequently Every course

Announcements

Assignments

Calendar

Chat

Content Viewer

Discussions

Drop Box

Email Instructor

Feedback

Forums

Glossary

Gradebook

Lessons

News

Polls

Quizzes & Tests

Resources

Sections


2012

70 | P a g e

Syllabus

Web Content (Link on left of site)

Wiki

Additional tool not in list above (please explain).

None

Finish this statement: I wish my professor(s) would make better use of ____________.

None

Have you used CLEW for group work?

Yes

No

What tool(s) did the instructor use for group-‐work? Select all that apply.

Discussions

Forums

Resources

Wiki

Calendar

Announcements

Other (please explain) ______________________

What feedback can you provide on your experience with group work?

None

How easy do you think CLEW is to use?

Very Difficult

Difficult

Moderate

Easy


2012

71 | P a g e

Very Easy

Other (please explain) ______________________

What would make CLEW easier for you to use?

Nothing, works fine as it is

My suggestion would be: ______________________

How do you rate CLEW on the following?

No opinion


Stability (Server up-‐time)

Reliability (does what it is supposed to do)

Usability (how easy user interface is to use)

Support for technical issues

Comments about your ratings above (optional).

None

Have you used other learning management systems?

Yes

No

Which system(s)? Check all that apply.

Blackboard Learn

Desire2Learn (D2L)

Moodle

WebCT

Angel

Other (please explain)

How does using CLEW compare to other systems you’ve used?

No opinion

Very Difficult


2012

72 | P a g e

Difficult

Moderate

Easy

Very Easy ______________________

Other / Comments (please explain)

Identify and rate your sources of support for CLEW (Check all that apply).

Not used


Help Desk

Your Professor

Your GA/TA

Online CLEW Help wiki (*New for Fall 2011)

“Your Comments” form on CLEW site

“Ask for help” form on CLEW site

LMS Administrator

Online videos

Learning Technologies Educational Consultant

Email: [email protected]

Comments on above rating(s).

None


2012

73 | P a g e

DEMOGRAPHICS With which Faculty are you primarily associated with?

Faculty of Arts & Social Sciences


Faculty of Graduate Studies

Faculty of Human Kinetics

Faculty of Nursing

Odette School of Business

Faculty of Science

Inter-‐Faculty Programs

Great Lakes Institute for Environmental Research

Faculty of Engineering

Faculty of Law


Rate your overall expertise with computers relative to other students at the University of Windsor.

Novice

Intermediate

Advanced

What is your year / status in your program?

First-‐year undergraduate (0-‐9 credits)

Second-‐year undergraduate (10-‐19 credits)

Third-‐year undergraduate (20-‐29 credits)

Fourth-‐year undergraduate (30-‐40 credits)

Professional student (e.g. Law, Nursing, etc.)

Masters student (e.g. MA, MFA, MBA etc.)

Doctoral student (e.g. PhD)

Other not mentioned above (please explain) ______________________


2012

74 | P a g e

APPENDIX 3: SUPPLEMENTARY CHARTS AND TABLES LMS STEERING COMMITTEE

Figure 30. LMS Steering Committee, 2011-‐2012

Chair, Vice-‐Provost Teaching and Learning

Chair, LMS Advisory Commiwee

Director, Teaching and Learning

Development, CTL

Actng Executve Director, ITS

Representatve, Office of the Registrar

Representatve, Leddy, Library

Manager, Educatonal

Technologies, CTL

Assistant Director, ITS


2012

75 | P a g e

INSTRUCTOR DISTRIBUTION BY FACULTY Table 20

Instructor distribution by Faculty

Faculty % of Faculty

Sessionals % of sessionals

Total % total

FASS 173 34.3 129 32.4 302 33.6 Education 25 4.9 90 22.6 115 12.8 Human Kinetics

21 4.2 5 1.3 26 2.9

Nursing 20 4 79 19.8 99 11.0 Odette School of Business

52 10.3 17 4.3 69 7.7

Science 113 22.4 37 9.3 150 16.7 Inter-‐Faculty Programs

0 0 3 0.8 3 0.3

Engineering 65 12.9 15 3.8 80 8.9 Law 32 6.4 23 5.8 55 6.1 Total 501 99.4 398 100.0 899 100.0

* GLIER – Great Lakes Institute for Environmental Research listed under Science. Graduate Studies faculty listed under their home Faculty.

Source: Faculty instructor data -‐ Office of Institutional Analysis


2012

76 | P a g e

FACULTY / SESSIONAL GENDER FISTRIBUTION FALL 2011 Table 21

HR Figures for Faculty / sessional distribution Fall 2011

HR Figures for Faculty / Sessional gender distribution Fall 2011 (estimated)

# Male % Male # Female % Female # Total % Total Sessionals 170 34.8 228 55.1 398 44.1 Faculty 318 65.2 186 44.9 504 55.9 Total 488 100.0 414 100.0 902 100.0 % Total 54.1 45.9 100.0 * Note: Some faculty are also sessionals

HR Figures for Staff # Male % Male # Female % Female # Total % Total Permanent/Full Time 357 74.5 559 63.2 916 67.2 Permanent/Part Time 77 16.1 217 24.5 294 21.6 Temporary/Full Time 8 1.7 26 2.9 34 2.5 Temporary/Part Time 37 7.7 83 9.4 120 8.8 Total 479 100.0 885 100.0 1364 100.0

% Total 35.1 64.9 100 Total

Number Staff

% of total

Total number of female instructor / staff

1299 57.3

Total number of male instructor / staff

967 42.7


2012

77 | P a g e

INSTRUCTOR / STAFF CLEW TOOL USE Table 22

Instructors -‐ [Tools] Which CLEW tools have you used in your courses?

Don't know


Announcements 0% 1% 5% 8% 13% 72%

Assignments 1% 20% 9% 11% 7% 52%

Calendar 2% 35% 24% 11% 10% 18%

Chat 3% 65% 18% 8% 0% 7%

Content Viewer 14% 47% 9% 9% 6% 14%

Discussions 0% 44% 16% 15% 13% 11%

Drop Box 4% 47% 11% 8% 8% 23%

Email Instructor 1% 24% 2% 10% 12% 50%

Feedback 5% 43% 11% 7% 7% 28%

Forums 4% 62% 13% 5% 9% 7%

Lessons 5% 34% 11% 8% 12% 30%

Glossary 9% 72% 12% 0% 3% 4%

Gradebook 5% 30% 4% 13% 16% 32%

News 8% 59% 12% 8% 3% 11%

Polls 8% 77% 7% 4% 1% 3%

Quizzes & Tests 3% 69% 8% 4% 11% 5%

Resources 0% 14% 2% 4% 12% 68%

Roster 1% 14% 6% 5% 14% 60%

Sections 12% 49% 8% 7% 1% 22%

Site Stats 4% 30% 14% 12% 8% 32%

Syllabus 1% 7% 1% 0% 16% 74%

Web Content 8% 43% 10% 10% 6% 22%

Wiki 12% 72% 7% 3% 0% 7%


2012

78 | P a g e

STUDENT CLEW TOOL USE Table 23 Students – [Tools] Which CLEW tools have you used in your courses?


Announcements 2% 3% 10% 30% 55%

Assignments 4% 7% 24% 40% 25%

Calendar 42% 27% 18% 8% 5%

Chat 49% 28% 16% 5% 2%

Content Viewer 30% 17% 24% 19% 10%

Discussions 24% 30% 28% 15% 4%

Drop Box 35% 23% 23% 14% 5%

Email Instructor 10% 13% 29% 25% 22%

Feedback 25% 15% 20% 25% 15%

Forums 45% 29% 16% 8% 2%

Glossary 75% 17% 6% 1% 1%

Gradebook 9% 8% 24% 37% 22%

Lessons 10% 7% 17% 39% 28%

News 43% 21% 18% 11% 7%

Polls 67% 22% 8% 2% 1%

Quizzes & Tests 37% 18% 21% 17% 7%

Resources 5% 5% 14% 37% 39%

Sections 40% 19% 19% 14% 9%

Syllabus 3% 4% 13% 29% 50%

Web Content (Link on left of site) 49% 20% 17% 9% 5%

Wiki 74% 17% 7% 2% 1%


2012

79 | P a g e

STUDENTS – TOOLS USED FOR GROUP WORK Response Chart Percentage

Discussions 86%

Forums 49%

Resources 37%

Wiki 6%

Calendar 6%

Announcements 45%

Other (please explain) 5%

Figure 31. Students – Tools used for group work


2012

80 | P a g e

References Arvan, L. (April-‐June 2009). "Dis-‐Integrating the LMS," EDUCAUSE Quarterly, vol. 32, no. 2 (April-‐June

2009). Contact North, (April, 2012) retrieved Apr. 20, 2012 http://www.contactnorth.ca/resources/learning-‐

management-‐systems-‐ontario-‐%E2%80%93-‐who-‐s-‐using-‐what Kuh, G., Kinzie, J., Schuh, J., Whitt, E. (2005). Student success in college: creating conditions

that matter. San Francisco: Jossey-Bass Mott, J. (2010). Envisioning the Post-‐LMS Era: The Open Learning Network, EDUCAUSE Quarterly

Magazine, vol. 33, no. 1, 2010 retrieved from Educause Quarterly website, April 2, 2012 http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/EnvisioningthePostLMSEraTheOpe/199389

Documents

CLEW!Survey!2011!!mediawikibe.uwindsor.ca › wiki › images › c › c1 › CLEWSurveyFinal.pdf · 2017-06-13 · CLEW!Survey!2011!! Students,!Instructors,!andStaff! Prepared!by!LorieStolarchuk,!LearningTechnologies!Educational!Consultant!