http://www.pages.drexel.edu/~jw65/
Engaging Students in Distance Learning
Jim WatersSusan Gasson
The iSchool at Drexel
Jim Waters, Susan Gasson Drexel University, 2009
If students have already paid us, why should we care if they are engaged ? Altruism – the long view? Pragmatism – they can always leave
taking their tuition money with them Student feedback and tenure decisions? Word gets out! – social networks
Students can vote with their virtual feet and Their very real wallets
Jim Waters, Susan Gasson Drexel University, 2009
Agenda How do I know if students are engaged? What is the effect of:
Question design? Course scaffolding? Instructor moderation? Being an entertaining instructor?
So what?
Jim Waters, Susan Gasson Drexel University, 2009
Research Study Analyzed course interactions via discussion board on
Blackboard learning system. 12 online MS courses (info. systems./info. Science) 313 Students, 11,497 messages Posts to discussion board + small group discussions
Analyzed Thread depth, thread length, participants Cognitive content of message Interactive intent of message Patterns of message sequences
Examined student outcomes related to interaction Pre and Post questionnaires Demographics and Attitudinal data
Jim Waters, Susan Gasson Drexel University, 2009
Three modes of learning engagement
Individual Participation Active Course Involvement Iterative Social Engagement
Fluid: students can move between modes reacting to drivers
Jim Waters, Susan Gasson Drexel University, 2009
Mode 1: Individual Participation The semi-transparent participant
Interacts with materials Internalizes knowledge Contractual obligation postings Broadcast messages Superficial learning Hermit!
Jim Waters, Susan Gasson Drexel University, 2009
Mode 2: Active Course Involvement Demonstrates (some) genuine interest
Interacts with peers (after a fashion) Translates community knowledge Relates posts to own experience or knowledge Internalizes community knowledge Ego-centric approach Small group or clique interactions
Jim Waters, Susan Gasson Drexel University, 2009
Mode 3: Interactive Social Engagement Motivated for interactive learning
Committed to greater group learning Interacts freely with peers Looped learning cycles Iterative internalizations/externalizations Social construction of knowledge
Jim Waters, Susan Gasson Drexel University, 2009
How can we tell what is going on?
Jim Waters, Susan Gasson Drexel University, 2009
Mode 1: Course Participation
How Do We Know A Project Is On Track? Instructor 10/21/07 2:36 PMRE:How Do We Know A Project Is On Track? S18 10/25/07 12:09 PMRE:How Do We Know A Project Is On Track? S10 10/25/07 6:29 PMRE:How Do We Know A Project Is On Track? S21 10/25/07 8:30 PMRE:How Do We Know A Project Is On Track? S17 10/26/07 7:38
PMRE:How Do We Know A Project Is On Track? S22 10/28/07 6:19 PMRE:How Do We Know A Project Is On Track? S2 10/28/07 7:04 PMRE:How Do We Know A Project Is On Track? S4 10/28/07 10:24 PMRE:How Do We Know A Project Is On Track? S10 10/28/07 10:26 PMRE:How Do We Know A Project Is On Track? S7 10/28/07 10:46 PMRE:How Do We Know A Project Is On Track? S8 10/29/07 12:59 AMRE:How Do We Know A Project Is On Track? S11 12/14/07 11:34 AM
Jim Waters, Susan Gasson Drexel University, 2009
Mode 1: Course Participation
Jim Waters, Susan Gasson Drexel University, 2009
Mode 2: Active Course InvolvementUnanswered questions Instructor 10/5/07 3:23 PM
RE:Unanswered questions S1 10/5/07 6:59 PMRE:Unanswered questions S17 10/6/07 3:41 AM
RE:RE:Unanswered questions S19 10/8/07 12:33 AMRE:RE:RE:Unanswered questions S20 10/8/07 10:52 AM
RE:Unanswered questions S13 10/6/07 10:52 AMRE:RE:Unanswered questions S18 10/7/07 4:14 PM
RE:RE:RE:Unanswered questions S6 10/9/07 9:45 PMRE:Unanswered questions S12 10/6/07 11:04 AM
RE:RE:Unanswered questions S20 10/7/07 10:34 AMRE:Unanswered questions S9 10/7/07 6:49 AMRE:Unanswered questions S21 10/7/07 4:36 PMRE:Unanswered questions S10 10/7/07 5:31 PMRE:Unanswered questions S4 10/7/07 10:59 PMRE:Unanswered questions S19 10/8/07 12:07 AMRE:Unanswered questions S8 10/8/07 2:21 PM
BOK as a communications/marketing tool S12 10/9/07 1:18 PMRE:Unanswered questions S8 10/8/07 2:48 PMRE:Unanswered questions S14 10/12/07 2:33 PM
RE:RE:Unanswered questions S12 10/12/07 3:22 PMRE:Unanswered questions S15 10/12/07 3:26
Jim Waters, Susan Gasson Drexel University, 2009
Mode 2: Active Course Involvement
Jim Waters, Susan Gasson Drexel University, 2009
Mode 3: Interactive Social Engagement
Jim Waters, Susan Gasson Drexel University, 2009
Mode 3: Interactive Social Engagement
Jim Waters, Susan Gasson Drexel University, 2009
What makes students engage at higher
levels in a distance-learning course?
Jim Waters, Susan Gasson Drexel University, 2009
Question Frequency vs. response
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
100.00LI
S-1
b
LIS
-1a
IS-3
IS-2
IS-4
IS-1
b
LIS
-2c
LIS
-2b
CO
M-1
b
CO
M-1
a
IS-1
a
LIS
-2a
Questions
Posts/question
Linear (Posts/question)
Jim Waters, Susan Gasson Drexel University, 2009
Question Design
I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [Cooking up a new project]
Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or slow]
I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [Fact-finding]
Jim Waters, Susan Gasson Drexel University, 2009
And the Winner is
I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [Fact-finding]
Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or slow]
I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [Cooking up a new project]
Jim Waters, Susan Gasson Drexel University, 2009
GoodI want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [Cooking up a new project]
150 posts Several sub-threads extremely deep (7 or 8 levels) Critique, feedback, support and facilitationWell-placed faculty moderation, nudges rather than cattle
prods Well-bounded but open-ended: students define problem Deliberately pitched as a cooperative task Concrete (well-defined) task Students negotiate the task meaning collaboratively
Jim Waters, Susan Gasson Drexel University, 2009
AverageI would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [Fact-finding]
85 postsModerate sub-thread depth (mostly 3 or 4 levels) 31% were messages from Instructor to students20% were messages from students to InstructorWell-placed faculty moderation, focus on challenging assumptions. Reasonably open-ended problem Far less cooperative inter-student activity Not pitched as a cooperative activity Students not answering a common question, but question is
defined
Jim Waters, Susan Gasson Drexel University, 2009
BadCritically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or slow]
46 postsLimited sub-thread depth - mostly 2 (question then single response)45% were messages from Instructor to students37% were messages from students to Instructor18% were student-student messages Faculty intervention much more critical (didactic) Five questions in one: 1 was open-ended 4 bounded Very little cooperative inter-student activity Not pitched as a cooperative activity
Jim Waters, Susan Gasson Drexel University, 2009
Good questions tended to be
Early rather than later First question in the week Early weeks better than later weeks
Open (scope), but bounded (problem structure) Permitted students to call upon their personal experience
with IT or organizations Permitted many ways to approach the issues
Negotiated rather than defined Permitted collaborative interpretation Allowed students to contribute by defining their own take
on the question. Relevance to students helps – war stories,
company policies and approaches
Jim Waters, Susan Gasson Drexel University, 2009
Bad questions tended to be
Following a highly-interactive question or later in the course Later questions much less interactive and constructive
across courses than earlier questions Cognitively complex
Containing multiple parts that needed to be considered in turn, or
Overly abstract, so students could not draw on their personal experience.
Socially isolating Fewer opportunities for interpretation and collaboration
in answering the question.
Jim Waters, Susan Gasson Drexel University, 2009
How does course scaffolding affect
student engagement?
Jim Waters, Susan Gasson Drexel University, 2009
Scaffolding Something to hang construction on Solid foundation for task
Materials Discussion Support for knowledgeable peers to contribute
Task requires an extension of prior abilities Provides a structure on which students can
build knowledge Task must be just beyond current unaided
skills but doable with help
Jim Waters, Susan Gasson Drexel University, 2009
Poor ScaffoldingYou've been asked to read the ALA Code of Ethics plus two other codes of ethics of your choice. What did you learn from this process? Did any common themes or concerns tend to emerge? What did you relate to in the ALA Code of Ethics? Were there things that seemed problematic, or that you disagreed with?
Codes of Ethics Instructor 1/28/08 3:15 PMRE: Codes of Ethics S5 1/31/08 7:00 AMRE: Codes of Ethics S15 1/31/08 5:49 PMRE: Codes of Ethics S13 1/31/08 9:17 PMRE: Codes of Ethics S14 2/1/08 12:05 AM
RE: Codes of Ethics S16 2/1/08 12:54 PM RE: Codes of Ethics S9 2/1/08 1:08 PM
RE: Codes of Ethics S17 2/3/08 1:20 PM RE: Codes of Ethics S11 2/2/08 3:26 PMRE: Codes of Ethics S18 2/3/08 4:23 PMRE: Codes of Ethics S19 2/4/08 5:33 PMRE: Codes of Ethics S6 2/5/08 6:24 PMRE: Codes of Ethics S1 2/5/08 11:03 PM
Jim Waters, Susan Gasson Drexel University, 2009
Better ScaffoldingCan ethical behavior really be codified by a professional organization?Can ethical behavior be enforced? How?
Jim Waters, Susan Gasson Drexel University, 2009
Unscaffolded Learning materials
A long list of codes of ethics web-pages Three abstract ethics articles
A body of solid material but this did not directly relate to the posted question or give a framework for answering the question
Jim Waters, Susan Gasson Drexel University, 2009
Scaffolded Learning materials
A description of ethical models Worksheet for ethical decision making
Actions and consequences Responsibilities and obligations A theoretical and pragmatic platform from which
discourse could be built Three sparse pages of bullet-points
Materials contained less information but provided a structure for thinking.
Jim Waters, Susan Gasson Drexel University, 2009
How does instructor moderation affect
student engagement?
Jim Waters, Susan Gasson Drexel University, 2009
Moderated or leave-alone ?
Two sections of an IS course delivered at the same time – same basic syllabus
~Same number of students (23/24) Selected six “identical” questions on each
section Different Instructor approach Heavy moderation vs. lightweight
moderation
Jim Waters, Susan Gasson Drexel University, 2009
Question Heavy Moderation
Low Moderation
Systems Analyst as problem solver 69 74
Agile methods 96 97Project design 150 97
Requirements Analysis 96 83Fact Finding 85 90Data Modeling Practice 182 180Average 112 103Tot messages 238 268Tot words 26270 57128 Average words/student
message110.38 213.16
Questions and Approach (messages)
Jim Waters, Susan Gasson Drexel University, 2009
Agile methods Heavy Moderation Low Moderation
Total Messages 96 97Instructor – student
messages 16 (17%) 0Student messages 80 97Student-student
messages 28 73Student-instructor
messages 52 24Deep thread messages
(students) 65 44Deep sub-threads ( 4
levels or greater) 10 8Deep sub-threads w/o
instructor intervention 2 8
Jim Waters, Susan Gasson Drexel University, 2009
Instructor interaction and student posts
0
500
1000
1500
2000
2500
INST-2
INST-4
INST-8
INST-8
INST-6
INST-8
INST-3
INST-4
INST-7
INST-3
INST-5
INST-1
Instructor DB postsStudent Posts
Jim Waters, Susan Gasson Drexel University, 2009
Are students more engaged when the
Professor is entertaining?
Jim Waters, Susan Gasson Drexel University, 2009
Deconstructing the Entertaining Professor Highly knowledgeable industry professional
Very Popular Instructor High level of interaction with students Regular internet chat presence Projects personality into discussion
High percentage of social interactions Voluntarily discusses hobbies, weather, music,
Disneyworld, cooking, children, Dickens, vintage cars, pets, gardening, insects, Star Wars, birds, Nintendo, Scrabble, foreign films, beer ….
Injects lots of jokes
Jim Waters, Susan Gasson Drexel University, 2009
Student participationStudents Discussion
Board Visits
Posts Topic Threads
Posts/Thread
23 24095 2745 67 40.9
Student Posts
StudentPosts/Thread
FormalQuestions
StudentPosts/Question
AverageThreadDepth per question
1648 24.7 30 67 8.6
Jim Waters, Susan Gasson Drexel University, 2009
Never mind the width, feel the quality
Threads show evaluation and analysis Some hypothesis formation Fairly advanced cognitive activity Some student-student interactionBut, some chaff among the wheat
5% Un-focused Anecdotes 32% fluff posts: “LOL,” “Awesome [dead rock star] story!”
“OMG,” “Pictures of gardening implements,” “Lawyer Jokes” 50% contribute to learning 33% student-Instructor messages
Still, pretty successful overall High student satisfaction Grades were comparable with Prof. Serious
Jim Waters, Susan Gasson Drexel University, 2009
Introducing Professor Serious
Same course – Same Syllabus Highly knowledgeable industry
professional Very skilled Instructor Low level of direct interaction with students Strong Topic-focus Little social interaction
Jim Waters, Susan Gasson Drexel University, 2009
Student participationStudents Discussion
Board Visits
Posts TopicThreads
Posts/Thread
24 13079 1458 352 4
Student Posts
StudentPosts/Thread
FormalQuestions
StudentPosts/Question
AverageThreadDepth per question
1334 3.79 14 95.3 5
Jim Waters, Susan Gasson Drexel University, 2009
Quality ? More collaborative learning Messages longer and more detailed Fairly advanced cognitive activity Much stronger student-student interaction
Stronger awareness of value of peer interactionsSo what?
High task-Focus 2.5% fluff posts 80% substantive knowledge-building posts 15% student-Instructor messages
Successful overall Moderate student satisfaction Grades were comparable with Prof. Entertaining
Jim Waters, Susan Gasson Drexel University, 2009
The tale of the tape Instructor participation of Prof. Entertaining
inflates thread depth slightly More posts, but smaller % focus on knowledge-building
Student-student posts more productive than student-instructor posts
Overall productive (knowledge building) activity was about the same for the two Professors
Student satisfaction slightly lower for Prof. Serious than Prof. Entertaining Fewer posts, but greater % focus on knowledge-building
But Stronger thought leaders for Prof. Serious Explicit kudos for peers in Prof. Serious
Jim Waters, Susan Gasson Drexel University, 2009
ConclusionsQuestion design can be crucial to
engagement Discussion needs to be framed as
collaborative not competitive Relevance to students helps – war stories,
company policies and approaches No payoff for frequency of questions
Course scaffolding aids engagement Focus and framing of questions Must support task in concrete manner
Jim Waters, Susan Gasson Drexel University, 2009
ConclusionsSuccessful course moderation hinges
on quality not quantity or frequency Knowing when to intervene Does not mean “being absent”
Being entertaining is not essential for success Tradeoff between popularity with students
and peer-engagement by students May shift focus from peers to instructor Excessive interaction is a lot of work
Jim Waters, Susan Gasson Drexel University, 2009
Students think they are learning more from peer-interaction
this was so helpful because often I was struggling with the same thing so I could learn from their errors and gain new information from the answers to their questions
I was moved to comment on how refreshing the lack of competition in the communications for the online classes seemed to me. It was a discussion and a sharing of experiences
Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback
No question that the on line discussion was critical to getting me through the class. There were mostly questions about how to..I've never done this before.
I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant.
Jim Waters, Susan Gasson Drexel University, 2009
Related References Waters, J. '
Social Network Behavior, Thought-Leaders and Knowledge Building In An Online Learning Community', Proceedings of Hawaii Intl. Conference on System Sciences (HICSS-41), Knowledge Management Track, Jan. 2008.
Gasson, S. and Waters, J. “How (not) to construct ALN course questions that encourage student participation in peer collaboration and knowledge construction,” 40th Hawaii International Conference on System Sciences, Hawaii, January 2007.
Waters, J., and Gasson, S. "Social Engagement in an Online Community of Inquiry," 27th International Conference on Information Systems (ICIS), Milwaukee WI, 2006.
Waters, J. “Determinants of Engagement in an Online Community of Inquiry,” The 12th Sloan-C International Conference on Online Learning, November 2006, http://www.sloanconsortium.org/conference/proceedings/2006/ppt/1162852287092.pot
Waters, J., and Gasson, S. "Strategies Employed By Participants In Virtual Learning Communities," Hawaii Intl. Conference on System Sciences (HICSS-38), Collaboration Systems and Technology track, IEEE Software Society, Hawaii, January 2005, p. 3b.
A full list of publications, with full copies of articles, is available at http://www.pages.drexel.edu/~jw65/publications.htm
Jim Waters, Susan Gasson Drexel University, 2009
Credits Dr. Susan Gasson
MBA, PhD from University of Warwick (Warwick Business School), UK. Prior to that, she worked in systems design , IS management, and IS software architecture
consultancy. Researches collaborative processes of design, problem-solving, learning. Associate Professor at Drexel University, Philadelphia (USA). Involved in online education since 2000(!).
Dr. Jim Waters BA in Psychology from Warwick University, UK (1979), MSc in Occupational Psychology from
University of Hertfordshire, UK (1991), MS in Information Systems (2002) and PhD (2009) from the iSchool at Drexel University.
Prior to that, he enjoyed a substantial career in systems design, management and IS consultancy.
Doctoral Candidate at the iSchool at Drexel University, graduating Summer 2009. Principal research interests lie in Online Collaborative Knowledge Building, Technology-Supported Learning, Student Role-Behavior in Online Learning Communities and HCI.
This material is based upon work supported by the National Science Foundation under Grant No. IIS-0347595. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.