An Affective Agent Playing Tic-Tac-Toe as Part of a
Healing Environment
Matthijs Pontier & Ghazanfar F. Siddiqui
Overview of this presentation
● Background● About the models incorporated in the agent● Application● Conclusions & Future research
Background
● Growing belief that not only healthcare, but also environment affects healing process
● Previous research:
Physical and social changes in environment can positively influence healing process Enhance recovery Reduce length of stay in hospital
Background
● Previous research: Poor design Negative effects:
● Anxiety● Delirium● Elevated blood pressure levels● Increased intake pain drugs
Unfamiliar environments Psychological stress that can negatively affect healing / wellness.
Background
● Loneliness is important contributing factor to Psychosocial stress
● Loneliness is common problem in long-term care facilities
● Previous research: Animal-shaped toys can be useful as a tool for
occupational therapy Paro is used for therapy at eldercare facilities, and
improves users’ moods, activeness and communicativeness
Animal-assisted therapy with AIBO robot dog helped just as good as with living dog
Background
AIBO Paro
Background
● Banks et al.: AIBO dog was not used to its full capacity, and results might be further enhanced
● If artificial companion demonstrates human-like emotional behavior, this might increase ability to reduce loneliness patients
● We present affective virtual agent that can play tic-tac-toe as a pilot application for this purpose Equipped with Silicon Coppélia: Integration of three affect-
related models. Therefore able to show human-like emotional behavior Simulated emotions: Joy, Distress, Hope, Fear, Surprise
The Models Incorporated in the Agent
● I-PEFiCADM: Model to let agents trade rational for affective choices, based on theory Frijda
The Models Incorporated in the Agent● EMA: Model to let agents exhibit and cope with
(negative) affect based on Smith & Lazarus’ theory
The Models Incorporated in the Agent
● CoMERG: Can simulate different emotion regulation strategies explained by Gross
Combined model: Silicon Coppélia
● Simulate affective decision making process: decisions based not only on rationality, but also on affective influences
● Simulate emotions based on beliefs about world-states, and how these affect goals
● Emotion regulation strategies can be applied to regulate (simulated) emotions
ENCODE COMPARE RESPOND
SITUATIONS FEATURES APPRAISAL APPRAISAL RESPONSEDOMAINS PROCESS COVERT OVERT
Inte
grat
ed m
ode
l
currentvalence
weightedfeatures
ethicsaffordances
aestheticsepistemics
currentstate
predicates
futurestate
predicates
affe
ctiv
e s
tate
s
“emotions”“mood”
Features are matched against goals, concerns, beliefs, intentions, etc. of self and others (allows taking perspectives)
involvementdistance
satisfa
ction
situationmodification
futurevalence
relevance
affe
ctive de
cision m
akin
g
situation selection
po
sitive ap
proach
neg
ative
app
roach
chan
ge
avoid
experiential behavioral
physiological(e.g., arousal, heart-beat, ……………., sweat, blush)
response modulation
cognitive change
attentional deployment
similarity
Relevance and valence (gray area) are the variables in the appraisal frames
use intentions
app
rais
al
fram
es
User feature
Affords Robot goal state
Positive / Negative
Facilitates / Inhibits
Desired / Undesired
Determining which action to take
● Agent has beliefs that actions facilitate goals (winning / losing) based on status tic-tac-toe game
● Agent can have desire to win, but also desire to lose● For action that facilitates desired goal, a high Action_Tendency
is generated● Positivity action based on belief action facilitates losing● Negativity action based on belief action facilitates winning● ExpectedSatisfaction(Action) =
wat * Action_Tendency + wpos * (1 - abs(positivity – biasI * Involvement)) + wneg * (1 - abs(negativity – biasD * Distance))
Calculating Hope and Fear
● Hope and fear are based on perceived likelihood [0, 1] of winning and losing (which are based on status tic-tac-toe game)
● Higher hope_for_goal if likelihood close to ‘f’● ‘f’ is set to 0.5● ‘Hope’ for negative ambition_level is fear
IF f >= likelihood THEN hope_for_goal =
-0,25 * ( cos( 1 / f * p * likelihood(goal) ) -1,5) * ambition_level(goal)
IF f < likelihood THEN hope_for_goal =
-0.25 * ( cos( 1 / (1-f) * p * (1-likelihood(goal)) ) -1.5) * ambition_level(goal)
Calculating Hope and Fear
● The following algorithm is performed:1. Sort hope_for_goal values in two lists: [01] and [0-1]
2. Start with 0 and take the mean of the value you have and the next value in the list. Continue until the list is finished. Do this for both the negative and the positive list.
3. Hope = Outcome positive list.
Fear = abs(Outcome negative list).
● This way, multiple hope_for_goals increase hope. ● However, with the more hope_for_goals there are, the
less impact each extra hope_for_goal has on the level of hope
Calculating Joy and Distress
● Joy and Distress are based on (not) reaching (un)desired goal-states (i.e., winning / losing):
● Reaching desired goal increases joy, decreases distress● Reaching undesired goal decreases joy, increases distress● Higher ambition level goal Bigger impact on emotions
IF ambition_level(goal) >= 0 THEN:new_joy = old_joy + mf_joy * ambition_level(goal) * (1-old_joy)
new_distress = old_distress + mf_distress * -ambition_level(goal) * old_distress
IF ambition_level(goal) < 0 THEN:new_joy = old_joy + mf_joy * ambition_level(goal) * old_joynew_distress = old_distress + mf_distress * -ambition_level(goal) * (1-
old_distress)
Calculating Surprise
● Agent generates expectations about move human or outcome game, based on status tic-tac-toe game
● Surprise = 1 – likelihood(move)● Surprise = 1 – likelihood(outcome_game)
Emotion expression by agent
● All 5 emotions are simulated in parallel, and are shown by the facial expression of the agent
● After each human move, the agent speaks a message depending on the level of surprise
● Satisfaction is calculated based on the outcome of the game and amount of money played for
● Relief = Satisfaction * Surprise● After finishing game, agent speaks message
based on Satisfaction and Relief
Agent 1: The agent tries to win
The ambition level for winning of the agent is set to 1
The ambition level for losing is set to -1
The weight of the affective influences in the decision making process is set to 0.
The agent always tries to win
Winning increases joy, and decreases distress
Losing decreases joy, and increases distress
The agent gets hope from a high likelihood to win
The agent gets fear from a high likelihood to lose
The agent becomes surprised from unexpected moves
Agent 2: The agent deliberately tries to lose
The agent has an ambition level for winning of -1
The agent has an ambition level for losing of 1
The weight of the affective influences in the decision making process is set to 0.
The agent always tries to lose.
Losing increases joy and decreases distress
Winning decreases joy and increases distress
The agent gets hope from a high likelihood to lose
The agent gets fear from a high likelihood to win
Agent 3: The agent decides whether it wants to win based on its money
The agent determines its ambition level for winning and losing on whether it has more money than the user or not.
If the user has more money than the agent, it will deliberately try to lose, but otherwise it will try to win.
Importance is determined by dividing the amount played for by the amount the agent has left
Importance determines the impact on joy and distress
If the outcome of the game is desired (agent wants to win, and wins, or wants to lose, and loses), joy is increased and distress is decreased
If the outcome of the game is undesired, joy is decreased and distress is increased
Agent 4: The agent is too involved with the user to win
The agent bases its decisions only on emotions
The agent is designed to be very involved with the user.
The agent performs actions that facilitate losing.
The agent has the ambition to win, but because it bases its decisions on emotions it tries to lose anyway
Losing decreases joy, and increases distress
Winning increases joy, and decreases distress
Agent 5: The agent is balanced, and wins sometimes, and loses sometimes
The agent bases its decisions partly on emotions, and partly on rationality.
The agent has always more ambition to win than to lose. How big this difference in ambition is, depends on the amount of money that is played for.
When playing for a high amount, the agent tries to win
When playing for a low amount, the agent is too involved with the human user to try to win
Losing decreases joy, and increases distress
Winning increases joy, and decreases distress
Conclusions
● We presented an agent equipped with a combination of emotion models, that can play tic-tac-toe
● Experimenting with various parameter settings indicates the affective agent shows realistic behavior
● This agent can be seen as a pilot application for an artificial interaction partner that can reduce loneliness
● In future research we intend to perform user studies to show whether this really is the case
● A link to the application can be found on my website:● http://www.few.vu.nl/~mpontier
Questions?