Feedback in International E-Learning Scenarios: Challenges and Solutions

Institut für Informatik und Wirtschaftsinformatik (ICB)

Learner’s Perceptions of Feedback: A Study across Five Countries Plymouth e-Learning Conference 2011 Thomas Richter

Picture: Winding-tower, Essen

Pictures: University of Duisburg Essen

Learners‘ Perceptions of Feedback: A Study across Five Countries

2

Open Discovery Space: European project as example for specific research situation

n  started April 2012, http://www.opendiscoveryspace.eu/ n  51 partners, 20 European countries, 14,3 Mio € budget n  Objective: Build up a Meta-OER-Portal for European Schools

o  transparently connecting 1.5 Mio open learning resources o  penetrate > 1% of all European schools o  motivate teachers and students to use the resources in their daily school work o  & support building local and international communities

How to ensure that courses in selected repositories that originally were designed for national contexts also are appropriate for

international contexts?


3

Previous Research: Obstacles in using learning contents from other contexts (OER) 1.  OPAL Project (EC funded LLP): International survey on Open

Educational Practices (500 samples) – Success factors and obstacles (online survey)1 o  Findings: One of the major challenges to adapt contents from other

(foreign) contexts is the missing awareness on possible (cultural) conflicting potential

2.  Investigation on German school teachers on special challenges related to the use of OER (group-interviews)2 o  Findings: German teachers almost exclusively use German learning

contents because they are not aware on how to determine cultural adaptation needs and thus, they fear to cause conflicts

3.  Study with e-Learning professionals in Germany, Austria, and Switzerland on special skills, learners need for e-Learning3 o  Findings: Self-motivation revealed being the most impacting ability

and motivation the core success-factor

1. Andrade, A., et al. (2011). Beyond OER – Shifting Focus to Open Educational Practices: OPAL Report 2011. Due-Publico, Essen. 2. Richter, T., & Ehlers, U.D. (2011). Barriers and Motivators for Using Open Educational Resources in Schools. eLearning Papers, No. 23, March 2011. 3. Richter, T. & Adelsberger, H.H. (2011). E-Learning: Education for Everyone? Special Requirements on Learners in Internet-based Learning Environments. In: T. Bastiaens & M. Ebner (Eds.), Proceedings of the World Conference on Educational Multimedia, Hyper-media and Telecommunications 2011, (Lisbon, Portugal), Chesapeake, VA: AACE, pp. 1598-1604


4

Open Questions …

What can educators do to foster the learners in online-scenarios to keep their motivation on the

necessary high level?

We at least can give feedback!

Consideration: If I know my local students, I can give them proper feedback, but … aren’t as well the demand for feedback as also the expected/required method/context culturally different? How to do it properly?


5

Feedback – Can it be wrong?

n  International distribution of learning resources meets national and thus, cultural borders

n  The consensus regarding “culture research” is that the concept of errors and the relationship to authorities have a cultural background1 … but what about perceptions of feedback?

n  Does the concept of errors in a society allow deducing if, how, and when learners want to receive feedback?

n  Investigated Culture-related aspects of Feedback: (various sources) o  When to give feedback (point of time?) o  Scope (item- or task-related?) o  Situation to give feedback (in group situations or individually?) o  Transmission code (explicit or implicit?)

-  Form (just evaluation? tell what’s wrong? How to do it better?) o  Is laud actually considered being motivating? o  Is critique motivating or discouraging? o  How about not providing any feedback?

1. Hofstede, G., Hofstede G. J. (2005). Cultures and Organizations. Intercultural Cooperation and Its Importance for Survival. USA, revised and expanded 2nd edition, McGraw-Hill Publishers.


6

Survey Questions (Feedback)

1.  When to give feedback? o  Directly when mistake found o  At the end of the task

2.  Scope/Situation o  Item related o  Task related o  Directly in group

3.  Transmission code o  Clearly point on errors o  Show how to do it better o  Just fair evaluation o  Show errors and explain why o  Point on errors but don’t explain why o  Show errors, explain why, and how to do it better

4.  Laud effects my motivation o  Positive o  Negative

5.  Critique effects my motivation o  Positive o  Negative

6.  I feel confused if there is no feedback


7

Survey Settings

n  Survey with university students from Bulgaria, Germany, South Korea, Turkey, Ukraine: July 2010-May 2012

n  Questionnaire using 4 point Likert-scale n  Only fully (100%) completed questionnaires were evaluated n  Settings per country:

o  Bulgarian students: online survey; language Bulgarian; link distributed through local students’ associations; 30 evaluated sample elements; response-rate unknown

o  German students (in-depth study): online survey; language German; full investigation in 3 universities; 1817 evaluated sample elements; response rate: 2-5%

o  South Korea (broad study): paper survey; random path algorithm for choice of sample-elements; students from 39 universities in and around Seoul; 286 evaluated sample elements; non-response rate 50%

o  Turkey: o  Ukraine:


8

Survey Results 1/2

!


9

Survey Results 2/2

Item Korea % (p/n)

Korea mean

Germany % (p/n)

Germany mean

4.a Laud positively effects my motivation

96.37 (p) 1.36 95.60 (p) 1.30

4.b Laud negatively effects my motivation

82.26 (n) 3.40 72.37 (n) 3.74

5.a Critique positively effects my motivation 86.69 (p) 1.88 63.24 (p) 2.29 5.b Critique negatively effects my motivation 65.73 (n) 2.92 57.68 (n) 2.80

6. I feel confused if no feedback is given 73.37 (p) 2.11 79.14 (p) 1.88

Agenda: 1 = fully agree 2= somehow agree 3= somehow disagree 4= disagree

Related to the topic „Feedback“, no significant differences between male and female students have been monitored.


10

Findings

n  Amazingly little differences have been found between the two contexts (Korea: errors = failure; Germany = a chance to learn).

n  In both contexts, as well laud as critique have positive effects on the motivation of the questioned students. In general, students in both contexts are confused if not receiving any feedback.

n  Students in both contexts expect and want to receive qualified feedback. Just pointing on errors or evaluating is not enough: The students in both contexts like to receive explanations on what the mistake is like and how to do it better.

n  Feedback should individually be given in the Korean context. In Germany, receiving feedback in the group-context is widely accepted.

n  German students prefer item-related feedback directly when the error has been found - this means we should monitor the “production process”. Korean students seem to accept both, feedback in the end of the task and during work (task solving strategies are culturally motivated).


11

Into the practice of E-Learning … conclusions (1/2)

n  Feedback is the strongest tool we have to keep learners on track. Frequently use it! Do not just criticise but also give the learners applause (laud): In E-Learning, they cannot see your reaction.

n  The majority of the participating students stated that they prefer qualified feedback. Just telling evaluation results as a number is not considered being sufficient.

n  Providing small tasks (e.g., multiple choice) within the LMS which automatically can be evaluated should be an added service (e.g., for self-control): When giving wrong answers learners should have a chance to repeat the task (Mason & Rennie 2006, p. 50). Also here, the correct answer should be provided if an error has been made (“correct answer feedback”, Fenrich 2005, pp. 185-187)

n  When giving feedback, it is meaningful to know about the culture of the recipients in order not to discourage the learners.

• Fenrich, P. (2005). Creating instructional multimedia solutions: practical guidelines for the real world. Informing Science Press. • Mason, R.; Rennie, F. (2006). Elearning: The key concepts. Routledge, USA, Canada.


12

Into the practice of E-Learning … conclusions (2/2)

n  Particularly in binational scenarios, implementing our survey is easy and the resulting awareness of possible cultural differences may prevent discouraging effects.

n  Qualified feedback (particularly if item-related) is difficult to be implemented as automatically given feedback: o  Reduce technology use: In E-Learning Situations, use E-Mail as primary

technology for feedback. Learners can store and come back to it. A lot of learners feel uncomfortable (overchallenged) using twitter & Co

o  If available, additionally chat is a great technology to discuss 1 on 1 how to do it better. Keep in mind that in intercultural situations, where at least one of the partners uses a foreign language, misunderstandings are easily produced. In chat, such can be determined much easier/faster.

n  Publicly present “typical errors” and discuss them in a forum, involving all learners. If the concept of errors does not allow to directly address the learner, don’t discuss with a particular learner but make general statements.


13

Thank you!

Thank you very much for your Attention!

Are there any questions?

(Also feel free to contact me after the session or via e-Mail)

thomas.richter @icb.uni-due.de