Affective UX: Challenges in UX involving affective computing

  • View
    665

  • Download
    3

Embed Size (px)

Text of Affective UX: Challenges in UX involving affective computing

Design experience first; design software afterwards:

Affective UXChallenges in UX design involving affective computing

Alfredo SnchezUniversidad de las Amricas Puebla

01WORLD IA DAY 2016PRESENTATION TITLE HEREHEADER OPTIONSub head or short descriptionSome kind of explanatory text, reference or footnote can go here and wrap to two lines, if needed.Some kind of illustration or image?

2006-2014, Alfredo Snchez

http://www.udlap.mx

Architecturing happinessHow do we know users are happy?

Can interactive systems detect happiness (or the lack thereof)?

Can interactive systems intentionally produce happinesss?

Can technology at least not get in the way? 2006-2016, Alfredo Snchez

AgendaTerms of affectionAffective computingDetectionSynthesisMediationAffect in UXRecent developmentOpen issues

Terms of affection

2006-2014, Alfredo Snchez

EmotionA reaction experienced usually toward a specific object and typically accompanied by physiological and behavioral changes in the body

2006-2014, Alfredo Snchez

MoodA conscious state of mind (typically lasting longer than an emotion)

joydepression 2006-2014, Alfredo Snchez

SentimentAn attitude or opinion a state that results from attributes assigned to objects, which are grouped into categories

2006-2014, Alfredo Snchez

The affect spectrumemotionmoodsentiment

may triggermay prompt

Duration( - ) ( + ) 2006-2014, Alfredo Snchez

9

Affective computing

2006-2014, Alfredo Snchez(Computer sytems) related to, arising from, or deliberately influencing affective phenomena

MIT Affective Computing Group

Affective ComputingRosalind Picard1995

Affective computing (for engineers) The study and development of systems and devices that can recognize, interpret, process, simulate and mediate human affects.

2006-2016, Alfredo Snchez

Why affective computing?

2006-2014, Alfredo Snchez

Why affective computing?

2006-2014, Alfredo SnchezI choose not to feelhttp://animala.mx/noticias/2015/02/27/adios-mr-spock/

Why Affective ComputingReason is, and ought to be, only the slave of the passionsDavid Hume, 1711-1776

Without passions we would lack all motivation, all impulse or drive to act, or even to reason 2006-2014, Alfredo Snchez

Why affective computing?A love/hate relationship with the computer and ITs in general 2006-2014, Alfredo Snchez

https://www.youtube.com/watch?v=ZcZBTB-dkqQ

Why affective computing? (for engineers) Studies by many researchers show a vital role of emotion in many processes:

PerceptionDecision makingCreativityComprehensionMemorySocial interaction 2006-2014, Alfredo Snchez

http://www.bintegra.com/wp-content/uploads/2012/05/Fotolia_socila_users_S.jpg

Some application areasEducation intelligent tutoringHealth monitoring patientsSafety monitoring drivers / pilotsSecurity detecting liesE-commerce attitudes towards products or servicesVideo games producing engaging environments, enriching UXUser studies incorporating emotion into evaluation

2006-2014, Alfredo Snchez

Why affective computing (for our community)Emotion is the way we communicate our experience 2006-2014, Alfredo Snchez

Main aspects of affective computing

perception

synthesismediation 2006-2014, Alfredo Snchez

(of emotions, moods and sentiments)

19

Affect detection

2006-2014, Alfredo Snchez

Emotion detection: From facial cues 2006-2014, Alfredo Snchez

http://npx.autismspot.com/facial_expression.jpg

Emotion detection: From postural cues 2006-2014, Alfredo Snchez

http://instridebodywork.com/wp-content/uploads/2013/05/postures-of-emotion1.jpg

Emotion detection: From voice cues 2006-2014, Alfredo Snchez

http://pandodaily.files.wordpress.com/2013/07/whisper-in-ear.jpg?w=584&h=329http://gigaom2.files.wordpress.com/2013/07/emotion-speech.jpg?w=300&h=200&crop=1

Emotion detection: From usage patterns 2006-2014, Alfredo Snchez

http://web.deepnetsecurity.com/products2/typesense.asp

Keystroke dynamicsNavigation patternsEye tracking

Emotion detection: From brain activity 2006-2014, Alfredo Snchez

Emotiv EPOC sensorHonda BCI system

Emotion detection: From sensors of autonomic activitySkin conductance increases even with a little surprise or anxiety

Blood volume/pressure increases with anger or stress

Respiratory rhythm varies with fear, sadness, happiness, anxiety, fear

Muscle tension increases with surprise and anxiety

2006-2014, Alfredo Snchez

Affectiva sensor

NeXus sensor

Emotion detection: From sensors of autonomic activity 2006-2014, Alfredo Snchez

Know your inner selfW/Me

Hack happinessFeel (by Sentio)

Some general techniques to support affective computingFor detection, synthesis and mediation 2006-2014, Alfredo Snchez

Facial Action Coding System (FACS) 2006-2016, Alfredo Snchez

http://www.theroadtosiliconvalley.com/tag/tim-roth/

FACSFacial Action Coding System (FACS), very popularDefines Action Units (AUs), contraction or relaxation of one or more face muscles

2006-2014, Alfredo Snchez

FACS (for universal emotions)

http://www.paulekman.com/facs/ 2006-2014, Alfredo Snchez

Posture, motion and gesture analysis 2006-2014, Alfredo Snchez

Posture analysis

http://people.ict.usc.edu/~gratch/CSCI534/ACII-Handbook-GestureSyn.pdf 2006-2014, Alfredo Snchez

Posture and motion analysisE.g. arm movements and emotion: 2006-2014, Alfredo Snchez

http://people.ict.usc.edu/~gratch/CSCI534/ACII-Handbook-GestureSyn.pdf

Gesture analysis 2006-2014, Alfredo Snchez

http://people.ict.usc.edu/~gratch/CSCI534/ACII-Handbook-GestureSyn.pdf

MPEG-4MPEG-4 (ISO144996) defines 86 Face Animation Parameters (FAPs)196 Body Animation Parameters (BAPs) 2006-2014, Alfredo Snchez

Some existing toolsFaceReader, SHORE, eMotion, RealEyes 2006-2014, Alfredo Snchez

Some challengesPrototypical vs. non-prototypical emotions Pure vs. combined emotionsMultiple subjectsPartly covered faces / bodiesTracking face or body movementLighting conditions

Integrating techniquesSupporting application development 2006-2014, Alfredo Snchez

Emotion in UX evaluation

2006-2014, Alfredo Snchez

Evaluating user experience with novel technologies impact of emotion (Gestures + contextual menus - video)

2006-2014, Alfredo Snchez[Magallanes, Molina, Mndez& Snchez 2012]

Heuristic evaluationProblems observed in actual use 2006-2014, Alfredo SnchezIssues identified by expertsIDIssueOccurrencesP1Movement of objects using two fingers.13P2Size of the keyboard.8P3Finding help for the user.5P4Finding the toolbar associated with each object.4P5Navigating using the fan menu.3P6Navigating between nodes.3P7Identifying the icon for linking to new pages 2P8Incomplete or inaccessible menus.1

IDTaskIssue1Insert two images, change their position and size.P12Enter a text next to the images as a caption.P23Obtain the toolbar for an image.P3, P44Pop a menu and discover how it works.P55Build a hierarchy of linked workspaces.P6, P76Get the menu in different positions. Cause the menu to go off the screen.P87Insert two images, change their position and size.P18Enter a text next to the images as a caption.P2

Additional feedback on emotions, based on Product Emotion Measurement Tool (PrEmo)

[Desmet 2002]Participants were asked to select an animated representation in correspondence with their emotional reactions

2006-2014, Alfredo Snchez[In collaboration with Universidad del Cauca, Colombia]

Positive emotions in spite of problems! 2006-2016, Alfredo Snchez

Positive emotions associated with novelty?Negative emotions associated with known interface elements?[Magallanes, Molina, Mndez& Snchez 2012]

ImplicationsTechniques for evaluation of usability and user experience are needed that take affect into account

Framework developed at UniCauca, Colombia[Mndez, Collazos, Granollers,Snchez 2014; Mndez 2015] 2006-2014, Alfredo Snchez

EmoValA framework for emotional evaluation designed to enrich existing UX evaluation methodsThree major componentsEmoVal-C: Behavioral evaluation (facial expressions, gaze, posture, movements)EmoVal-S: Self-report of emotions while using a systemEmoVal-F: Physiological sensing, mainly brain activity during system usageSupporting website (http://artemisa.unicauca.edu.co/~ymendal/EmoVal) 2006-2016, Alfredo Snchez

[Mndez 2015]

EmoVal: Detailed support / facial expressions 2006-2016, Alfredo Snchez

[Mndez 2015]

EmoVal: Detailed support / posture and movement 2006-2016, Alfredo Snchez

[Mndez 2015]

Building capabilities for affect detection

2006-2014, Alfredo Snchez

Current scenario: Extending an app to include more techniques is not straightforward

Application domainTechniqueTechnologies 2006-2014, Alfredo Snchez

Goal: Integration platform

2006-2014, Alfredo Snchez[Snchez et al. 2015]

Architecture for implementation

2006-2014, Alfredo Snchez[Snchez et al. 2015]

Simple data model and messaging protocol

2006-2014, Alfredo Snchez[Snchez et al. 2015]

Prototypical apps: Kinect-based emotion detection 2006-2014, Alfredo Snchez

14 sorpresa[Snchez et al. 2015]

Prototypical apps: Kinect-based emotion detection 2006-2014, Alfredo Snchez[Snchez et al. 2015]

Prototype evaluation

2006-2014, Alfredo Snchez[Snchez et al.