SMS Q&A Course Evaluations Stephen Marquard
[email protected] http://twitter.com/stephenmarquard
Centre for Educational Technology, University of Cape Town June
2010
Slide 2
3 Sakai contrib tools SMS Mobile messaging from Sakai Q&A
Questions and Answers Course Evaluations Online course evaluations
and surveys 211th Sakai Conference - June 15-17, 2010
Slide 3
SMS 11th Sakai Conference - June 15-17, 20103 Wth Sakai U 2 can
snd txt msgs to grps of ppl! W00t!
Slide 4
SMS Overview What is it? SMS provides mobile text messaging in
Sakai, including a tool for sending messages, and a switchboard for
handing incoming messages to tools or scripts. Where do I find it?
http://confluence.sakaiproject.org/display/SMS/
http://confluence.sakaiproject.org/display/SMS/
https://source.sakaiproject.org/contrib/sms/
http://jira.sakaiproject.org/browse/SMS More information?
Screenshots on Confluence. Ask questions on the sakai-dev list, or
contact UCT. 411th Sakai Conference - June 15-17, 2010
Slide 5
Outgoing SMS 11th Sakai Conference - June 15-17, 20105 In the
SMS Messages tool, a.Write message (with context and sender)
b.Select recipients by one or more of: Users with a role (e.g.
Student) Users in a group (e.g. Tutorial 1) Users by name List of
mobile numbers c. Calculate cost / credits d. Send e. View delivery
status See more screenshots at
http://confluence.sakaiproject.org/display/SMS/SMS+
Messages+Screenshots
Slide 6
Incoming SMS Examples 11th Sakai Conference - June 15-17, 20106
Placement offers: UCT Health Sciences offers you a place in MBChB
for 2010: SMS offer studentnumber accept/decline to ##### (200+
responses in 10 minutes). Q&A: What new use of Vula are you
considering? SMS your answer to ##### R 3461 I would like to use
Vula for surveys Q&A: Ask a question about Vula: SMS your
question to #####: Q smsdemo How can I use Vula for surveys? Other
possible uses for incoming SMS: Chat, Polls, Glossaries, etc.
Slide 7
Incoming SMS Any tool/service can accept incoming articles by
registering a command prefix with the SMS service. Either implement
ShortMessageCommand or configure a REST endpoint for incoming
messages. Example
code:https://source.sakaiproject.org/contrib/qna/trunk/impl/
src/java/org/sakaiproject/qna/logic/impl/sms/QuestionSmsC
ommand.javahttps://source.sakaiproject.org/contrib/qna/trunk/impl/
src/java/org/sakaiproject/qna/logic/impl/sms/QuestionSmsC
ommand.java Example REST registration: see
http://confluence.sakaiproject.org/display/SMS/Incoming+S MS
http://confluence.sakaiproject.org/display/SMS/Incoming+S MS 11th
Sakai Conference - June 15-17, 20107
Slide 8
Deployment considerations Service provider You need a service
provider with an SMPP gateway (e.g. http://www.clickatell.com).
What destination networks / countries are supported, and at what
cost per message?http://www.clickatell.com Mobile numbers Populate
user numbers in the user profile, or implement a provider (e.g. for
LDAP lookup or some external source). Privacy and defaults Opt-in
or opt-out, private or visible. Cost and accounting model Who
incurs costs for sending messages? Prepaid or postpaid? Default
implementation supports a prepaid model, with credit allocation per
site. 11th Sakai Conference - June 15-17, 20108
Slide 9
SMS: Privacy Our approach at UCT (Vula = our Sakai): Student,
staff and third party cell numbers in Vula come from Peoplesoft,
SAP HR, Third Party system Default My Workspace / Profile settings:
Allow Vula users to send SMS messages to my cellphone: Yes Hide my
cellphone number from other Vula users: Yes Users can opt out (0.5%
so far) People sending messages see only the username, not the
users cell number Use SMS wisely Otherwise people will opt out
which makes it less useful for everyone.
Slide 10
Q&A 11th Sakai Conference - June 15-17, 201010
Slide 11
Q&A Overview What is it? A tool which supports
question-driven interaction (optionally anonymous) with web, email
and SMS workflows (comparisons: Yahoo Answers, StackOverflow,
Purdue Hotseat). Where do I find it?
http://confluence.sakaiproject.org/display/QNA/Home
http://confluence.sakaiproject.org/display/QNA/Home
https://source.sakaiproject.org/contrib/qna/ More information? A
question is worth a thousand answers (Paris Sakai conference
presentation):http://confluence.sakaiproject.org/download/attachments/39616
558/paris-sakai-qna.pdfhttp://confluence.sakaiproject.org/download/attachments/39616
558/paris-sakai-qna.pdf Ask questions on the sakai-dev list, or
contact UCT. 1111th Sakai Conference - June 15-17, 2010
Slide 12
Why questions? Many students ask the same question Students
will ask a question online which they wouldnt or couldnt ask
face-to-face Students may ask a question differently (or at all) if
they can ask it anonymously Students answer each others questions
The student questions and answers provide a window into student
thought processes, their difficulties, confusions, misconceptions
and successes. Questions and answers provide academic support, peer
support, and diagnostic insights.
Slide 13
Supported workflows Instructor-led: Student asks question >
instructor responds privately (by email), and/or publicly (in the
site) Collaborative: Student asks question > peers respond >
(optional) instructor / TA provides or nominates a definitive
answer Questions may be moderated, i.e. only become visible in the
site once reviewed by an instructor / TA Site members may be
allowed to ask questions anonymously
Slide 14
Tool demo Set up tool personality and workflow through Options
and Permissions View, ask and respond to questions Create and
organise categories and reorganize questions across categories
Slide 15
Slide 16
View questions and ask a new one
Slide 17
Viewing responses
Slide 18
Respond to a question (instructor)
Slide 19
Add an answer (student)
Slide 20
Organise questions
Slide 21
SMS support Q&A is minimally SMS-enabled Supports asking
questions by SMS, receiving answers and/or retrieving answers Not
really exposed in UI yet (except for Options)
Slide 22
Course Evaluation 11th Sakai Conference - June 15-17,
201022
Slide 23
Course Evaluation Overview What is it? A tool for delivering
course evaluations or surveys. Where do I find it?
http://confluence.sakaiproject.org/display/EVALSYS/Home
https://source.sakaiproject.org/contrib/evaluation/ More
information? Ask on the [email protected] mailing
[email protected] UCT screencast
(instructor-managed evaluations)
http://confluence.sakaiproject.org/display/EVALSYS/Course+Eva
luations+video+tutorial
http://confluence.sakaiproject.org/display/EVALSYS/Course+Eva
luations+video+tutorial 2311th Sakai Conference - June 15-17,
2010
Slide 24
Caveats How you might implement this depends heavily on
institutional processes and policy. Evaluation System is a complex
tool which has support for different institutional models of
delivering evaluations. The information in Confluence is not
up-to-date. No 2 sites run the tool in exactly the same way. Ask on
the evaluations list! 11th Sakai Conference - June 15-17,
201024
Slide 25
Generic issues and concerns Response rates may be lower
(dependent on lecturer involvement and feedback) but students give
more written feedback online than in paper-based evaluations Moving
from paper to online does not lead to less favourable ratings (U.
Michigan) The set of respondents is representative of the course
cohort by race, gender, final course mark: lower response rates do
not skew results (IPD, UCT) Online evaluation is popular with
students and provides more opportunities for student feedback
Different opinions and strategies about incentives What happens to
the results?
Slide 26
UCT self-service model Site Owners or Support Staff can create
and assign evaluations to one or more of their Vula sites. The
evaluation owner decides: o Who is evaluated (lecturers and tutors)
o The questions o Start and end dates o How to encourage students
to respond o What to do with the results (subject to Department or
Faculty processes or guidelines)
Slide 27
UCT possible teaching models One or more lecturers (aka
instructors) o Every student evaluates all lecturers in the
evaluation o Students are taught in cohorts: each student chooses
his/her lecturer to evaluate Tutors (aka TAs) o Tutors interact
with students as a group: students evaluate the support of the
tutors in general o Every student evaluates all tutors individually
o Each student belongs to one or more tutor groups, and therefore
chooses one or more tutors to evaluate
Slide 28
UCT: running an online evaluation 1.Add the Course Evaluations
tool to one or more existing Vula course sites 2.Create a new
evaluation: a)Choose a template to start from b)Choose start and
end dates c)Choose reminder options d)Assign the evaluation to one
or more Vula sites e)Identify Lecturers and Tutors to be included
in the evaluation f)If necessary, edit the evaluation (add, remove
or change questions) 3.While the evaluation is open: a)Look at the
response rates, and encourage students to complete the evaluation
b)Optionally, extend the closing date if the response rate is
insufficient 4.When the evaluation has closed: a)View results
(online, PDF, export to Excel) b)Provide feedback to students
and/or class reps c)Provide a copy of results to HoD, other
staff
Slide 29
UCT evaluation structure About the course About you (About all
lectures/tutors) Lecturer 1 questions Lecturer question a Lecturer
question b, Lecturer 2 questions Lecturer question a Lecturer
question b, Tutor 1 questions Tutor question a Tutor question b,
Tutor 2 questions Tutor question a Tutor question b, General
questions Lecturer questions repeated for each assigned lecturer
[respond to all, select one, select many] Tutor questions repeated
for each assigned tutor [respond to all, select one, select
many]