Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

DETECTING EMOTION MODEL IN E-LEARNING SYSTEM

GUEY-SHYA CHEN, MIN-FENG LEE

Graduate Institute of Educational Measurement and Statistics, National Taichung University of Education, Taichung, Taiwan

E-MAIL: grace@mail.ntcu.edu.tw.antoni0656l@gmail.com

Abstract:

Affective computing is computing that relates to human

affects things. In this research, it proposed a teaching model

with affective computing. It uses a novel method to detect

learner's emotion and adjust emotion when learner's emotion

without in positive emotion status. The detecting emotion

teaching model uses emotion management module that include

the detecting emotion and emotion map functions to detect

learner's emotion and record emotion status for learning. This

research uses emotion map to record the emotion locus for

learning activity. In this detecting emotion teaching model

integrates learning activities and emotion locus to create a

complete learning portfolio. And it can be applied in analyzing

learning status for adjusting learner's situation. By this

research detecting emotion teaching model it makes a method

that was based on learner's emotion to build a more effective

learning environment. The detect emotion teaching model,

which is a kind of innovative learning model can be applied in

game based learning for continuously developing in the future

and it can be used in a variety of teaching environment for

increasing study effect.

Keywords:

Affective computing; E-Iearning; Detecting Emotion;

Emotion map

1. Introduction

E-learning has developed and built for many education organizations so rapidly in the recent years. Students can use the web account to learn the courses online freely in their own time. In the general case, the course of E-learning platform can be composed two mainly materials, html and video. There are not only teaching materials but also include test to assess each leamer's achievement in the course [1]. Sheryl mentioned that "as e-learning courses proliferate, there are increasing levels of individual learner autonomy and control in such contexts" [2]. Vikas Joshi

978-1-4673-1487-9/12/$31.00 ©2012 IEEE

wrote that the biggest challenge in e-Learning today is to keep learner interested and enthusiastic through the entire duration of the course. This can be achieved through high cogmtlVe level interactions, use of games, active videos/audios, puzzles, simulation and so on. However, sometimes this is compromised due to time and cost constraints during course creation [3].

In the traditional Western society, it used dichotomy to separate reason and emotion, which was inherited from Descartes' dualist vision of the mind and body, in the 20 century little attention focused on the role of the affectivity in cognition and learning. The real class environment seems concentrate on the cognitive capacities of the students and makes a fitness knowledge system for themselves [4]. The recent researches of psychologists and neurologists point out the major key of the motivation and the affectivity in cognitive activities, especially learning [5, 6]. Psychologists and teachers mentioned that the emotions affect learning [7].

Owing to the major key of the affectivity in learning, the researchers of computer in education domain have brought up new approach in educational techniques in order to change the e-learning systems more customized also for the affective states of learner.

This research focuses on leamer's emotion status in learning progress. Learners interact with detecting emotion interface in the learning activity interval and the emoticon which detects the emotion while replying is an uncomplicated tool for communication. Emotion orbit data is not only for once measurement but also amasses and outlines for tracing individual learning emotion area.

2. Research related literatures

2.1. Affective Computing

The professor, Minsky (1985) mentioned that "The question is not whether intelligent machines can have anv

1686

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

emotions, but whether machines can be intelligent without any emotions." [8, 9] The researcher, Picard (1997) defines affective computing as "computing that relates to, arises from or deliberately influences emotions" [10]. Affective computing is an interdisciplinary field spanning computer sciences, psychology, and cognitive science [11]. Picard (1997) described that an affective computing system must have a few of the following capacities: 1. recognize, 2. express, or 3. Possess emotions. It focuses on creating personal computing systems having ability to sense, recognize and understand significantly after the positive interventions than the conditions with no intervention [12]. There are several ways of current study to measure for emotion status.

• Automatic detection of cognitive-affective states by computers [13, 14]

• Recent progress in real time affects detection [15] • Progress on affect detection through body movement

and gestures [16] • Through acoustic-prosodic cues [17], lexical features

[18] and physiological features [19]

2.2. Emotion

Emotion was defined in Wikipedia that is a complex psychophysiological experience of an individual's state of mind as interacting with biochemical (internal) and environmental (external) influences [20]. In humans, emotion fundamentally involves "physiological arousal, expressive behaviors, and conscious experience. " The theories of emotions were developed for a long time from the Ancient Greek Stoics as well as Plato and Aristotle. And different philosophers mention it such as Rene Descartes, Baruch Spinoza and David Hume.

The defmition from oxford dictionary online, emotion is a strong feeling deriving from one's circumstances, mood, or relationships with others [21]. The Famous emotion theory, James-Lange theory of emotion, that the bodily changes follow directly the perception of the exciting fact, and that our feeling of the same changes as they occur is the emotion. Common sense says they lose their fortune, are sorry and weep; they meet a bear, are frightened and run; they are insulted by a rival, are angry and strike. The hypothesis here to be defended that this order of sequence is incorrect and the more rational statement is that we feel sorry because they cry, angry because they strike, afraid because they tremble ... Without the bodily states following on the perception, the latter would be purely cognitive in form, pale, colorless, destitute of emotional warmth. They might then see the bear, and judge it best to run, receive the

insult and deem it right to strike, but we should not actually feel afraid or angry [21].

Emotion is the complex inside processing of an individual's state of mind. Leland Beaumont states that every emotion includes mood, motivation, temperament, personality, and disposition [22].

James-Lange theory was proposed by William James, in the article 'What is an Emotion?' [23]. In James-Lange theory argued that emotional experience is largely due to the experience of bodily changes. Another psychologist Carl Lange also proposed a similar mechanism in emotion reaction. As James says "the perception of bodily changes as they occur is the emotion. "

2.3. Emotion classification

Emotion classification was written in Wikipedia: "Many theorists define some emotions as basic where others are complex. Basic emotions are claimed to be biologically fixed, innate and as a result universal to all humans and many animals as well. Complex emotions are then either refined versions of basic emotions, culturally specific or idiosyncratic." [24] The basic categories of emotion are joy, angry, sad, and happy. The categories of complex emotions can be combined or associated from basic emotions in emotions conditioning.

In Table 1 , Emotion classifications collected from early scholars. Since ancient times, many scholars proposed their emotion classifications. The researchers used their aspect to define the emotion expression making many varieties in emotion classification.

TABLE 1. DIFFERENT EMOTIONAL CLASSIFICATION FROM SCHOLAR (SCHMITTE,2010)

Scholar

Rene Descartes (1956-1690) Baruch de Spinoza (1632-1677) Thomas Hobbes (1588-1679) A. Jorgensen (1894-1977) Silvan Solomon Tomkins (1911-1991)

Emotion Classification

love, hate, desire, joy, sadness and admiration

joy, sorrow and desire

appetite, desire, love, aversion,

hate, joy, and grief fear, happiness, sorrow, want,

and angry shyness

interest, surprise, joy, anguish,

fear, shame, disgnst, and rage

In 1972, Ekman devised a list of basic emotions that are not culturally determined, but universal. The six basic emotions are: 1. Anger, 2. Disgust, 3. Fear, 4. Happiness, 5. Sadness, and 6. Surprise. Ekman expanded the basic emotions including a range of positive and negative

1687

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

emotions in 1990. The newly included emotions are: Amusement, Contempt, Contentment, Embarrassment, Excitement, Guilt, Pride in achievement, Relief, Satisfaction, Sensory pleasure, Shame [25].

2.4. Circumplex Model of Affect

How to record the emotion in every unforgettable moment? These emotions of the moment are the treasures. In this research, it not only captures the location of picture in every moment but also keeps the emotion of the every serendipitous moment. In 1980, James proposed A Cirucmplex Model of Affect. This model was offered both as a way psychologists can represent the structure of affective experience, as assessed through self-report, and as a representation of the cognitive structure that laymen utilize in conceptualizing affect. The Circumplex Model of Affect is a spatial model based on dimensions of affect that are interrelated in a very methodical fashion [26]. Affective concepts fall in a circle in the following order: pleasure (0° ), excitement (45°), arousal (90°), distress (l35°), displeasure (180°), depression (225°), sleepiness (270°), and relaxation (315°) shown in Figure 1.

In the Circumplex model of affect, there are scaling 28 emotion denoting adjectives in four different ways.

90°

.� • A5.10N'SHEO

• �XCJTfP iolARMI:O.

AFUtD��iI'· A�ED.

OCSlWst.t> •

fftl4'JlATfO • • U<lIGHT�!>

• HA!'f'I 1�0 ______ 4_� ____ .... /)0 .. ,LEA$E:D

MrSE RA&tE • • <X.AO

80RfU •

� .. "Rf!> •• �t<lPY

• • n

Figure 1. Direct circular scaliug coordinates for 28 affect words[26J

2.5. Positive Psychology

Positive psychology was found by Seligman and Csikszentmihalyi. They wrote: "We believe that a psychology of positive human functioning will arise, which achieves a scientific understanding and effective interventions to build thriving in individuals, families, and

communities. " [27] Hence, they point towards the importance of personal strengths such as creativity, courage, perseverance, kindness, and fairness [28, 29]. Positive psychologists seek "to find and nurture genius and talent ", and "to make normal life more fulfilling ", not simply to treat mental illness [30]. Positive psychology focuses the idea that learners can use their own personal qualities to optimally action in their life environment, so that their actions are both effective and personally fulfilling. Acting upon these qualities leads to what Csikszentmihalyi calls flow [31], a concept that comes obviously close to the concept of existence, as Csikszentmihalyi states it as a state of totally being in the here-and-now, optimally connecting the demands of the situation with one's inner capacities [32]. The researchers in this field refer to positive psychology can be discriminated into three overlapping areas of research: [29]

• The Pleasant Life, or the "life of enjoyment " • The Good Life, or the "life of engagement " • The Meaningful Life, or "life of affiliation "

Seligman also explains positive emotion including 3 categories [33].

• Past positive emotion: satisfaction, well-being, thanking, forgiving, forgetting

• Present positive emotion: pleasure, flow • Future positive emotion: optimism, hope, faith, trust

Past positive emotion was presented via the way of gratitude or forgiveness to the satisfaction or well-being. There are two emotions of present positive emotion and present in pleasure or flow. Pleasure emotion can be easily for achieved. Flow emotion can give the individual the satisfaction in a long time. Individual has future positive emotion that can be confront the challenge with reliance and confident.

3. Method

Detecting emotion model in learning system is a circulating instruction framework by learner's emotion status to adjust teaching strategy. This model arranges instruction progress via learner's emotion status.

3.l. Learning Flow

Learners get into the learning model that practices the first time detecting emotion. The learning system analyzes the learner's emotion status and arranges the instructing materials via first time emotion status. The learning system

1688

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

keeps the learner in an appropriate emotion to practice learning lesson. The detecting emotion process in every learning unit provides leamer's emotion status for learning system analyzing and arranging learning progress.

The learner must be in the positive emotion area to practice the lessons. The learner will practice the emotion adjust process while their emotion in an inappropriate status. Therefore, there are instruction lesson, emotion detection process, emotion adjusting process and examination in every instruction unit that forms a circulating instruction framework (Figure 2).

3.2.

Learner

Fbsi t i ve BIDt i on hea?

Yes

Fbsi t i ve BIDt i on Pr ea? >---N::>------.I EnIDt i � o���st i ng

Yes

Figure 2. Learner's Learniug Flow

Detecting Learning Emotion

The emotion detection interface gather the leamer's emotion information passing to the learning system for detecting emotion and analyzing. This function computes the emotion area and records emotion value.

3.2.1. Emotion Detection Process

The emotion detection process gathers the emotion information by emoticon from emotion detection interface and transforms the emoticon to the emotion numerical value for calculating emotion area. This process uses the numerical emotion area value to classify the emotion classification. It uses the emoticon that comes from leamer's interaction information to connect with Circumplex model of affect. There are 28 emotion classifications in Circumplex model of affect make the relation to emoticons (see Table2).

TABLE 2. EMOTICON (FROM: WINDOWS MESSENGER)

Emoticon

3.2.2. Emotion Detection Interface

This interface uses situational interaction questions to obtain leamer's responses for calculating emotion area value and recording emotion status. The detection emotion interface interacts directly with leamer. This interface is an important role in the detecting emotion module (see Figure 3). This interface use the emoticon to gather the intuitively response from learner uses situational interaction questions. The emoticon can be transformed the emotion classification via the emotion management unit. By the leamer's emotion status, instruction unit adjusts the teaching materials.

.---------------. I Emoticon I , , 1 ______ -------_ ..

Emotion Management Emotion Detection Emotion Map

Instruction Unit

Figure 3. Detecting emotion module

1689

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

3.2.3. Emotion Map

Every point that can be used in locating, tracing, outline personal positive emotion area on emotion map (see Figure 4) is a leamer's emotion feedback from learning process. In addition, personal positive emotion area difference that takes the information to the system for analyzing instruction arrangement can be drawn in emotion map. The personal portfolio integrates emotion map can show the whole learning path of individual that can trace the personal learning state for learning diagnosis.

"I!EIY

4.

Alarmad

"'fr�ai�gry 'Aim Oiatre88e

. . nee Aroused • 'Astonl8hed

... --- ... f'ruelr alBd -­

L ming er1J.C1i�f1

'Mlserable

• Sad 'OIoomy

, , ,

"

,

'Dapreesed -Bored Droopy' Tire�

• Sleepy

, Positive +J'ppy

, ,,"

f>ISlBSd

'----- , Olad

Serene At e/l8lt. "Cl?nlBnl

• SaU81i8d Calm Relaxed --

Figure 4. Emotion Map

Conclusions

R.EA9JRE

In this detecting emotion model aims to intensify the effect for instruction on learning system. The emotion detection interface of learning system via intuitively response from learner uses situational interaction questions. In this response interface makes interaction with learner. It makes the learning system more user-friendly and avoids suspending for learning. Emotion is a unique issue in this research, because learners always have all kinds of affect in every teaching activity. It is collecting emotion information during teaching process for suggestion instruction plan. From this model, emotion data of learner responding can be used in 2 functions: (1) tracing learning process in every instructing stage, (2) transform emotional fluctuating data to emotion map for calculating personalize learning emotion area. The emotion map is not only collecting the leamer's emotion but also analyzing the individual positive learning area for constructing a fitting learning path.

4.1. Further work

This detecting emotion model tries to use the intuitively response from learner for obtaining emotion data. This is not only method for obtaining leamer's emotion data. Many researchers attempt to obtaining or measuring emotion in various approaches, for example: facial expression, speech intonation, or brain waves by EEG. In the further work, this detecting emotion model will consider to integrate the above-mentioned methods for accurate emotion measuring in instructing process.

References

[1] H. Kuo-An and Y. Chia-Hao, "Assessment of affective state in distance learning based on image detection by using fuzzy fusion, " Knowledge-Based Systems, vol. 22, pp. 256-260, 2009.

[2] S. Sharma, G. Dick, W. Chin, and L. P. W. Land, "Self-Regulation and E-Learning, " presented at the Fifteenth European Conference on Information Systems, 2007.

[3] V. Joshi. (2007). on Only Jazz Up, Create Interactive

e-Learning Rapidly? Available:

http://www.digitallearning.iniarticles/article-details.as p?articleid= 1315&typ=CORPORA TE%20DIAR Y

[4] P. A. Jaques and R. M. Vicari, "A BDI approach to infer student's emotions in an intelligent learning environment, " Computers &amp; Education, vol. 49, pp. 360-384, 2007.

[5] A. Damasio, Descartes' Error: Emotion, Reason and

the Human Brain: Random House, 2008. [6] C. E. Izard, J. Kagan, and R. B. Zajonc, Emotions,

cognition, and behavior: Cambridge University Press, 1988.

[7] D. Goleman, EQ: Emotional intelligence. New York: Bantam Books, 1996.

[8] M. Minsky, The society of mind: Simon & Schuster, 1988.

[9] M. Minsky, The emotion machine: commensense

thinking, artificial intelligence, and the future of the

human mind: Simon & Schuster, 2006. [10] R. W. Picard, Affective Computing: MIT Press, 2000. [11] J. Tao and T. Tan, "Affective Computing: A Review, "

in Affective Computing and Intelligent Interaction

First International Conference, Beijing, China, 2005, pp. 981-995.

[12] I. M. Rezazadeh, X. Wang, M. Firoozabadi, and M. R. Hashemi Golpayegani, "Using affective

1690

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

human-machine interface to increase the operation performance in virtual construction crane training system: A novel approach, " Automation in

Construction, vol. 20, pp. 289-298, 20 II. [13] S. D'Mello, P. Chipman, and A. Graesser, "Posture as

a predictor of learner's affective engagement, " Proceedings of the 29th Annual Cognitive Science

Society, 2007. [14] A. Kapoor, W. Burleson, and R. W. Picard,

"Automatic prediction of frustration, " International

Journal of Human-Computer Studies, vol. 65, pp. 724-736, 2007.

[15] M. Pantic and L. 1. M. Rothkrantz, "Toward an affect-sensitive multimodal human-computer interaction, " Proceedings of the IEEE vol. 91, pp. 1370 - 1390, 2003.

[16] G. Castellano, S. D. Villalba, and A. Camurri, "Recognizing human emotions from body movment and gesture dynamics., " in Affective computing and

intelligent interaction: second international

conference, ACII 2007, Lisbon, Portugal, September

12-14, 2007 : proceedings, A. Paiva, R. Prada, and R. W. Picard, Eds., ed: Springer, 2007.

[17] M. Grimm, K. Kroschel, H. Harris, C. Nass, B. Schuller, G. Rigoll, and T. Moosmayr, "On the necessity and feasibility of detecting driver's emotional state while driving., " in Affective computing

and intelligent interaction: second international

conference, ACII 2007, Lisbon, Portugal, September

12-14, 2007 : proceedings, A. Paiva, R. Prada, and R. W. Picard, Eds., ed: Springer, 2007.

[18] J. Wagner, T. Vogt, and E. Andre, "A systematic comparison of different HMM designs for emotion recognition from acted and spontaneous speech., " in Affective computing and intelligent interaction: second

international conference, ACII 2007, Lisbon, Portugal,

September 12-14, 2007 : proceedings, A. Paiva, R. Prada, and R. W. Picard, Eds., ed: Springer, 2007, pp. 114-125.

[19] T. Komatsu, S. Ohtsuka, and K. Ueda, "Comprehension of users' subjective interaction states during their interaction with an artificial agent by means of heard rate variability index., " in Affective

computing and intelligent interaction: second

international conference, ACII 2007, Lisbon, Portugal,

September 12-14, 2007 : proceedings, A. Paiva, R. Prada, and R. W. Picard, Eds., ed: Springer, 2007, pp. 266-277.

[20] D. G. Myers, Psychology: Worth Publishers, 2009. [21] P. C. Ellsworth, " William James and emotion: Is a

century of fame worth a century of misunderstanding?, " Psychological Review, vol. 101, pp. 222-229, 1994.

[22] M. Hofmann and L. Beaumont, "Chapter 7 -Interactive Content De1ivery- Instant Messaging, " in Content Networking, ed San Francisco: Morgan Kaufmann, 2005, pp. 179-216.

[23] W. James, What Is an Emotion?: Wilder Publications, 2007.

[24] Wikipedia. Basic and Complex Emotions. Available: http://en.wikipedia.org/wikilEmotion_classification

[25] P. Ekman, "Basic Emotions, " in Handbook of

Cognition and Emotion, T. Dalgleish, M. 1. Power, and M. Power, Eds., ed: John Wiley & Sons, 2000.

[26] 1. A. Russell, "A Circumplex Model of Affect, " Journal of Personality and Social Psychology, vol. 39, pp. 1161-1178, 1980.

[27] M. E. P. Seligman and M. Csikszentmihalyi, "Positive psychology: an introduction., " American Psychologist,

vol. 55, pp. 5-14, 2000. [28] C. Peterson, A primer in positive psychology. New

York: Oxford University Press., 2006. [29] M. E. P. Seligman, "Positive psychology, positive

prevention, and positive therapy., " in Handbook of

positive psychology, C. R. Snyder and S. 1. Lopez, Eds., ed: Oxford University Press, 2005, pp. 3-9.

[30] B. Benard, Resiliency: what we have learned: WestEd, 2003.

[31] M. Csikszentmihalyi, Flow: The Psychology of

Optimal Experience: HarperCollins, 2008. [32] P. C. Meijer, F. A. 1. Korthagen, and A. Vasalos,

"Supporting presence in teacher education: The connection between the personal and professional aspects of teaching, " Teaching and Teacher Education,

vol. 25, pp. 297-308, 2009. [33] M. E. P. Seligman, Authentic happiness: using the new

positive psychology to realize your potential for

lasting fulfillment: Free Press, 2002.

1691