Application of Emotion Recognition Methodsin Automotive Research

Matthias Wimmer, Christian Peter, Jörg Voskamp, Martin A. Tischler

Institut für Informatik

2nd Workshop “Emotion and Computing”, 10.9.2007, Osnabrück

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

2

Theoretical Background of Driving Pleasure

Driver‘s Objective: Optimal Activation

Task Demands

Driveability

Sportiveness

Comfort

Capability

Task DifficultyResources und

Restrictions

Veh

icle

Environment

Emotional estimation

Transfer ofDriver‘s Request

Jordan (2000): “Four Pleasures” Physio-pleasure Psycho-pleasure

Socio-pleasure Ideo-pleasure

based onFuller (2005)

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

3

Measurement of Emotions in Vehicles

EMOTION

Subjectivecomponent

Expressivecomponent

Behavioralcomponent

Verbal report Driving dynamics

Physiologicalcomponent

Cognitivecomponent

Video/Audio-recording

Measurement Verbal report

Facial expressions

Gestures

Changes in voice

ECG, EMG, EDAQuestioning during driving

Driving style and operator control action

Cognitive appraisal of the vehicle and the situation

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

4

Pilot Study „Driving Pleasure W204“

2007 C220 CDI (W204) 1982 190E (W201)

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

5

Cooperation

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

6

Subjects: 8 non professional drivers (age: 33-53)

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

7

Task:Drive on three courses

Bosch Proofing Ground Boxberg(near Heilbronn)

Autobahn

Country road

Handling course

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

8

Technical Setup

driving dynamics

meterEREC glove

video camera

microphone

cell phonehandsfree set EREC

recording unit

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

9

The Methods in Detail

Physiological measurement system EREC (skin resistance, heart rate, temperature)

Interviews and questionnaires after

driving each car

Video and audio recorder on the

back seat

Drivers were asked to “think aloud”

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

10

Subjective Driving Experience

1,0 3,0 5,0 7,0

subjective control

no worries about safety

drove too risky

vehicle is predictable

vehicle met my desires

MB 190EMB C220

strongly agreestrongly disagree

N=8, Method: questionnaire

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

11

Facial Expression RecognitionInstitut für Informatik

Mercedes-Benz 190EMean expression of happiness: 4,2% Peak value: 15,0%

Mercedes-Benz C-Class 220Mean expression of happiness: 5,5% Peak value: 19,8%

0% = neutral face100% = maximum expression

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

12

1

1 3 5 7 9

A

C

H

A H C

MB 190E

MB C-Class 220

A AutobahnC Country roadH Handling

Valence

Activation

Speech Analysis

+

+

-

-

Most important parameters• changes of pitch• intensity• energy changes over frequency bands

Average confidence1.1-1.2 for valence0.9-1.0 for arousal

Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"

13

Conclusions

Methods show corresponding results

Speech analysis

Easy to apply, but speech is not a natural product while driving

Facial expression recognition

Challenge in fitting the face model and classifying the facial expression

Problems: changing light and background, non emotional-related head movements, driver is talking

Advantages of tested methods

Non disturbing

Continuous data collection

Useful completion to psychological methods