Upload
janis-carson
View
212
Download
0
Tags:
Embed Size (px)
Citation preview
Evaluating the Impact of a VLE on Learning and Teaching?
Dr Barbara NewlandLearning Design Studio
Academic ServicesBournemouth University
http://www.bournemouth.ac.uk/lds/[email protected]
Overview Aims of evaluation Durham context Methodology Results Conclusions
Aims of evaluation International widespread
adoption of VLEs Longitudinal study of impact
of a VLE on learning and teaching
Qualitative and quantitative methods
University of Durham 670 academic and 280
academic-related staff 9400 undergraduates
and 2400 postgraduates Founded in 1832 Split campus Research led MIS – outsourced to
Unisys Learning Technologies
Team (LTT)
Images of Durham
Learning Technologies Team aims to encourage, support and develop
elearning throughout the University by: providing advice, information and training supporting the University's learning
environment - duo (Durham University Online)
developing interactive materials 4 members of staff – Juliette Pavey, Kate
Boardman, Malcolm Murray, Victoria Boyd
Duo Durham University Online http://duo.dur.ac.uk
Uptake of duoEstimated use by Xmas 2000
Actual use by Xmas 2000
Use at June 2003
200 students3 departments
4286 students21 departments211 modules
10,000 students700 academics700 modules
Implementation Varied patterns of departmental
implementation All courses All first year courses Individual academics
Administration courses Duo is now embedded into
learning and teaching
Methodology for duo surveys Quantitative and qualitative Online surveys Analyse factors such as gender, year of
study and faculty 3 faculties – Arts and Humanities, Science,
Social Science and Health Relate to C&IT skills in learning and
teaching Data collected from students and staff
in all faculties and both campuses
Response rate All students and staff are in duo so hard
to know the response rate in relation to active users which has increased over the years
Staff April 01 (n = 91)May 02 (n = 153)May 03 (n = 283, c. 40%)
Students May 01 (n = 793)May 02 (n = 2435)May 03 (n = 3668, c. 37%)
Survey questions Similar questions for staff and students
in all 3 years to enable comparisons : between years between staff and students
Software ease of use levels of confidence features used
Survey questions Frequency of use and access
during term time and vacations Learning and teaching
contribution effect
Student responses – support, frequency and access Help documents
19%, 15%, 10% Small and decreasing use
Accessed duo at least once a week 70%, 71%, 71% Consistent pattern but increasing numbers over time
Accessed duo out of term time 60%, 61%, 64% Consistent pattern but increasing numbers over time
Students: Overall, how do you rate the contribution duo has made to your learning?
6.5
53.1
26.6
2.1 1.4
0
10
20
30
40
50
60
%
excellent good nodifference
poor very poor
Contribution
Staff: Overall, how do you rate the effectiveness of duo as a learning resource?
16.3
48.1
11.7
3.2 1.1
0
10
20
30
40
50
60
%
excellent good average poor very poor
effectiveness
Staff: Has duo affected your approach to teaching?
10.90.58.6
80
My basic approach to teaching seems to be changing, in part because of duo
My basic approach to teaching has not changed, but duo helps me do certain things better
There is minimal potential for change
I can't teach as w ell w hen I rely on using duo
91.6
8.4
yes no
Students: If duo is not currently being used for all your modules, would you recommend that it should be?
Staff use of duo features - content 62%, 49% staff have used more features 79%, 76% staff have developed content
since the previous year Course information – 74%, 84%, 92% Course documents – 77%, 89%, 95% External links – 42%, 56%, 55% Bibliography – 35%, 53%, 43%
(plus 17%, 21%, 21% link to OPAC) Staff information – 49%, 63%, 72%
Staff use of duo features - communication Group work – 25%, 23%, 26% Email – 72%, 77%, 80% Announcements – 75%, 84%, 86% Discussion board – 16%, 32%, 31% Chat – 10%, 4%, 6%
Staff use of duo features - assessment Assignments – 53%, 63%, 67% Quizzes – 11%, 20%, 26% Surveys - 15%, 13%, 21% Digital drop box – 10%, 6%, 15%
Reflect current ways of teaching Context – relevant, flexible (start from
where they are at) Collis also found at the University of
Twente that “instructors are most likely to begin by choosing aspects of a system that reflect their current ways of teaching, and then gradually move to new instructional approaches and new features.” (Collis and Messing, 2001)
Lecture notes - staff Is it good practice to put lecture
notes online? Depends on what the lecturer does
with them 2/3 put lecture summaries, handouts,
full notes and/or Powerpoint slides after the lecture
Lecture notes - students Depends how students use them
73% listen more 81% clarify information 72% improve lectures 93% study for exams
2/3 (64%, 64%) students note-taking had stayed the same but for ¼ (27%, 26%) it had decreased
Only 1%, 1% stated no longer attended lectures
Gender No statistical significance between
gender and Staff with effectiveness as learning
resource Students with contribution to learning Both with levels of confidence
Higher percentage of females think duo helps them to manage their learning
Staff by faculty – skills More Science faculty found it very
easy/easy to use compared to Arts Some Arts staff took longer to gain
confidence (6-10 visits) Fewer Science staff chose to have
training Reflects the levels of staff
confidence in C&IT skills found in Audit June 2000
Staff by faculty – teaching 2002 - no significant difference 2003
Basic approach is changing Science and social science 12% compared to
3% arts Basic approach has not changed but
duo helps me do certain things better Science 79%, social science 76% compared
to 98% arts
Students by faculty Contribution that duo has made to their
learning overall 2002 - no significant difference 2003 – higher percentage of science (69%)
and social science (52%) rate “good” or “excellent” compared to 42% arts students
Higher percentage of social science students think duo helps them to manage their learning “greatly” or “quite a lot” compared to science students
Students by year group Contribution that duo has made to
their learning overall 2001 and 2002 - 2nd and 3rd year
students rates more higher than 1st years (1st years have always had duo)
2003 – 1st year students rate slightly higher than 2nd and 3rd years
C&IT skills survey of new undergraduates – 2002 n = 2656 82% of students perceive themselves to
be confident/very confident computer users
60% have brought a computer with them to University
69% have been using a computer for 5+ years (including 21% for over 10 years) and only 2% have been using a computer for <2 years
C&IT skills survey of new undergraduates - 2002 79% perceive themselves to have
good/advanced email skills 65% perceive themselves to have
good/advanced web browser skills 70% stated a preference for a
combination of web and paper based for learning support materials
C&IT skills survey of new undergraduates - 2002 95% own a mobile phone 5% have a PDA 38% take part in online discussions 59% take part in online chat 49% have a qualification in IT 54% regularly switch between several
applications when using a computer
C&IT skills survey of academic staff - 2000 n = 576 (53%) 90% perceive themselves to have
good/advanced email skills 67% perceive themselves to have
good/advanced web browser skills only 17% of staff had taken part in online
discussions only 12% of staff had taken part in online
chat
Catalyst Introduction encouraged
continual reflection on teaching Postgraduate Certificate in HE
compulsory for new academics from Jan 2001
Academics working together within courses
Factors influencing enhancement How used How integrated into curriculum Quality of the parts eg content,
assessment, communication (earlier research)
Enhance learning if the parts enhance learning
Flexibility allows lecturers to choose parts
Is the whole greater than the sum of its parts? But is a VLE more than the pieces
put together?
Evaluating the impact of a VLE on learning and teaching Incredible uptake of duo which has become
embedded into the University Staff and students believe duo has enhanced
teaching and learning Academics are gradually using more features
so this may result in more interactive and innovative teaching
Students expectations are increasing as the basic use of a VLE is assumed
Further analysis is being undertaken
References Collis, B. and Messing, J. (2001) Usage, attitudes and
workload implications for a web-based learning environment, Association of Learning Technology Journal 9 (1)
Collis, B. and Moonen, J. (2001) Flexible Learning in a Digital World, London, Kogan Page
Laurillard, D. (1993). Rethinking University Teaching: a Framework for the Effective Use of Educational Technology. London: Routledge.
Richardson, J., & Turner, A. (2000). A large-scale ‘local’ evaluation of students’ learning experiences using virtual learning environments. Educational Technology and Society, 3 (4).
Salmon, G. (2000). E-moderating: the Key to Teaching and Learning Online. London: Kogan Page.