On Shape and the Computability of Emotions - Wang Group:...

On Shape and the Computability of EmotionsXin Lu, Poonam Suryanarayan, Reginald Adams, Jia Li, Michelle Newman, James Z Wang

The Pennsylvania State University, University Park, PA

SummaryWe mainly investigate the impact of shape featuresin natural images on the emotions aroused in hu-man beings.

• We studied shapes and its characteristicslike roundness, angularity, simplicity, andcomplexity to understand the emotional re-sponses of human beings.

• Image features used were mainly inspired bystudies in visual arts and psychology.

• We model emotions from a dimensional per-spective using valence and arousal measure-ment.

• We also try to focus on the challenging prob-lem of distinguishing images with strongemotional content from images which evokeweak emotions in human beings.

IAPS DatasetInternational Affective Picture System (IAPS) 2008dataset consists of about 1193 pictures which havebeen rated by both male and female subjects ona scale of 1-9 on the Valence, Arousal and Domi-nance content.

Example images from IAPS

Shape FeaturesRoundness and Complexity of shapes have beenwell studied in psychology which indicate rounderand simpler images are more pleasing than other-wise. These attributes can be measured using

• Line Segments and their orientation, lengthand mass

• Contiguous Lines and their degree of curv-ing, length span, line count and mass

• Angular lines and their discrete angle counts

• Curves and their fitness, circularity, area, ori-entation, mass and representative curves

Perceptual shapes were extracted using contourextraction techniques from Arbelaez et al.

Images and their characteristic shape features

Images with largest and smallest number of angles.

0 2 4 6 8 10 12 14 16 18 200

0.5

1

1.5

2

2.5

3

0 2 4 6 8 10 12 14 16 18 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16 18 200

0.5

1

1.5

2

2.5

3

3.5

4

0 2 4 6 8 10 12 14 16 18 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16 18 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

The distribution of angles in images.

Images with highest and lowest degree of curving

Emotion ModellingEmotion can be modelled from both dimensionalas well as discrete emotional perspective.The basic dimensions of emotion representationare:

• Valence - Intrinsic positiveness or attractive-ness of the image

• Arousal - Represents how soothing the im-age is

• Dominance - The magnitude of the emotionin the image

These dimensions can be used to represent discreteemotions as well. We use eight emotion categorieswhich have four positive and four negative valencemeasures - anger, disgust, fear, sadness, amuse-ment, awe, contentment and excitement.

Emotions in Valence-Arousal-Dominance Space

ExperimentsIn order to show the strength of shape features weperform the following tasks and compare the accu-racies obtained by shape features, Machadjik et al.(Color, texture, etc.) and combination of both.

• Predicting the Valence and Arousal Values

• Classifying 8 discrete emotion categories

• Classifying images with and without strongemotions

ResultsSVM with RBF Kernel was used for both classifica-tion as well as regression modelling.

0

0.425

0.85

1.275

1.7

Shape Jana’s Feature All Features

Valance Arousal

Mean Square Error in the prediction of valanceand arousal measurement

0.27

0.283

0.295

0.308

0.32

shape Jana’s features all features

Average accuracies for the eight classclassification task

70

72

74

76

78

Shape Jana’s Feature All Features

Set 1 Set 2

Classification accuracy for emotional images andneutral images with forward selection strategy

References[1] J. Machajdik and A. Hanbury. Affectiveimage classification using features inspired bypsychology and art theory. In ACM MultimediaConference, 2010.

[2] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik.Contour detection and hierarchical image segmen-tation. IEEE Transactions on Pattern Analysis andMachine Intelligence, 2011.

1