Embed Size (px)
Citation preview
Do’s and Don’ts for the Software Companion
David Traum USC Institute for Creative Technologies!(First presented at ACL Workshop on Companionable Dialogue Systems July 15, 2010)!
2!
Outline
What is a companion? !– And what is not!
10 Doʼs and Donʼts for Software companions!
3!
What is a companion?
Wikipedia: Ladyʼs companion!– A lady's companion was not regarded as a servant!– The companion's role was to spend her time with her employer, providing
company and conversation, to help her to entertain guests and often to accompany her to social events. !
Heroʼs companion!– Wikipedia: sidekick!
A sidekick is a close companion who is generally regarded as subordinate to the one he accompanies. !
Some well-known fictional sidekicks are Don Quixote's Sancho Panza, Sherlock Holmes' Doctor Watson, The Lone Ranger's Tonto, Batman's partner Robin,!
4!
What do you call the person that a companion is a companion to?
Not satisfied with!– Owner?!– User?!– Master?!– Employer?!– Charge?!
Perhaps!– Principal!
5!
Key aspects of Companions
Relationship!– Unquestionable Loyalty!
Always faithful to principal! Always available when needed!
– 2nd banana, sidekick! Knows when to speak and when to be silent! Mixed Initiative!
– Preference for principal!– Interacts (differently) with!
Principal! Other! Group interaction!
– Long-term! Multiple episodes! Grows with principal!
Motivation!– Partial overlap in skill sets!
Some similarities! Some differences!
– Some deficits!– Some advantages!
– Range of activities! Social interaction! Cooperative tasks! Information exchange! Task assignment!
6!
Gilgamesh & Enkidu (first companion?)
7!
Don Quixote & Sancho Panza
8!
Robinson Crusoe & Friday
9!
Pinoccio & Jiminy Cricket
10!
Sherlock Holmes & Dr Watson
11!
Lone Ranger & Tonto
12!
Companion Role
A companion is not a/an!– Enemy!– Rival/competitor (while a companion)!– Comrade!– Leader!– Teacher/sage/mentor!– Invisible/imaginary friend!– Servant!– Slave!– Tool!
13!
Batman & Robin
14!
Not a companion: Imaginary Friend
15!
Not a companion: love interest
16!
Not a companion: authority/rival/associate/comrade
17!
Rick & Sam (Companion)
18!
Who is a companion?
19!
Han & Chewie
20!
Companions?
21!
Non-companions
Mentor! Comrade! Quest Goal!
22!
10 Companion do’s and don’ts
1. Do something useful!2. Donʼt just do one thing !3. Do understand the
situation (as much as possible!
4. Do what I want!
5. Be a good conversationalist!
6. Revisit conversations appropriately!
7. Do understand what you are saying!
8. Donʼt be repetitive!9. Do learn and customize!
10. Maintain immersion!
23!
1: Do Something Useful
Task-oriented dialogue systems!– Services, e.g.!
Flight/train booking! In-car !
– music selection!– Navigation!– Email browsing!
– Information providing, e.g.! Flight/train/bus inquiry! Weather inquiry! Movie listings! Bank balance!
24!
1b: Don’t just be a tool
Do it as a companion!– Agent with goals, plan recognition, !
Not blind script following!– Negotiate, come to best collaborative decision!
Point out problems with goals! Point out problems with approaches! Suggest options and solutions!
– Give in if necessary (after making the case for other options, if appropriate)!
25!
Focus=1!Lt: U9 “secure a landing zone”!Committed(lt,7), 7 authorized,Obl(sgt,U9)!Sgt: U10 “first we should secure the assembly area”!Disparaged(sgt, 7), endorsed(sgt,2)!Lt: U11“secure the area”!Committed(lt,2), 2 authorized,Obl(sgt,U11)!Sgt: U12“yes sir”!Committed(sgt,2), Push(2,focus)!Goal7:Announce(2,{1sldr,2sldr,3sldr,4sldr})!Goal8: Start-conversation(sgt, ,{1sldr,2sldr,…},2)!Goal8 -> Sgt: U21 “Squad leaders listen up!”!Goal7 -> U22 “I want 360 degree security”!Push(3, focus)!Goal9:authorize 3!Goal9 -> u23“1st squad take 12-4”!Committed(sgt,3), 3 authorized!Pop(3), Push(4)!Goal10: authorize 4!Goal10 -> u24“2nd squad take 4-8”!Committed(sgt,4), 4 authorized!Pop(4)!…!A10: Squads move!A10: grounds U21-U26,…! ends conversation about 2, realizes 2!Focus = 7!
Example: MRE Task-Related Dialogue
Render Aid
Secure Area
Secure 12-4
Secure 8-12 Secure Accident
Secure 4-8
Squads in area A=Lt, R=Sgt
A=Sgt,R=1sldr
A=Sgt,R=2sldr
A=Sgt,R=4sldr A=Sgt,R=3sldr
Area Secure
1
2 3
4
5 6
Decomposition
Decomposition Secure LZ
A=Lt ,R=S
7
26!
2: Don’t just do one thing
Need to work together on a range of tasks! Need to know which one to focus on when (Yang & Heeman)!
Need to learn new tasks !– e.g. Allen, et al. (2007). PLOW: A collaborative task learning agent. Named
Best Paper, National Conference on Artificial Intelligence (AAAI). Vancouver, BC.!
27!
3: Understand the situation (as much as possible
Multimodal perception!– How many speakers!
Participant roles!– Ongoing (non-linguistic) tasks!– Locus of attention (gaze, orientation)!
28!
4: Do what I want (you to do)
Tell me stuff I might like to know!– When Iʼm receptive!
Might need to wait and reintroduce topic later!– When it’s urgent!
Do things for me!– When I ask!– When you are *sure* that I will like it!
Don’t interrupt me with irrelevant or unimportant issues!– Don’t be a paperclip!
29!
Not now clippy! Do I really look like I don’t know what I’m doing?
Wait ‘till Iʼm stuck! Or bring it up when I’m not so busy.!
30!
5: Be a good conversationalist
Be interesting! Have a range of conversational moves and strategies!
Have a number of different types of things to say! Be a good listener!
31!
Chatbots
http://www.alicebot.org/
32!
5a: Be Interesting
Tell narratives!– Tell stories!– Relay news!– Self-disclosures!– Jokes!– Gossip (Brusk 2008, Brusk et al, Sigdial 2010)!
Donʼt monopolize the conversation!– Donʼt drone on!– Be interactive!
Give Leading information (prompts), ! Assess follow ups! Continue/elaborate when prompted or when shown interest!
– Know when to shut up!
Donʼt be too interesting !– Itʼs about your principal, not about you the companion!
33!
5b: Be a good listener
Listen and understand user narratives!– Classify type and respond appropriately!
Attentiveness (backchannels)! Sympathy/Empathy! Prompts for next move! Clarification/elaboration requests! Assessments! follow-ups!
– Be attentive to what people want to talk about!
Remember things for the future!
Donʼt be afraid to admit when you donʼt understand!– Donʼt force the issue, give up when appropriate!
34!
MRE/SASO Taking Initiative Model
What to communicate!– Task model!– Emotion model!– Special domain-specific
rules!
When to communicate!– Response only!– Too much silence!– Too much
misunderstanding!– Too much irrelevance!– Directed by other!
How to communicate!– Questions!– Hints!– Suggestions!– Performances!
35!
6: Revisit conversations appropriately
Don’t just repeat the same story in the same way to the same person! Reminisce over interesting things when relevant!
Integrate new information with similar previous topics!– perspective!
36!
7: Understand what you are saying
Be able to do Bi-directional processing (be able to interpret what you say – as a listener would)!
Remember what you said! Be capable of meta-language about your utterances!
– Be able to understand and answer “why” and “how” questions!– Be able to give motivation, evidence and rationalizations for what you say!
Or be prepared to drop it.!!
37!
8: Don’t be repetitive
Donʼt repeat the same utterance verbatim! Have different ways to say the same thing!
– Lexical choice!– Syntactic, rhetorical, and interactive structures!– Summarization/elaboration!– Speech acts/politeness!– prosody!
Donʼt ask the same query over and over!– Even when you donʼt understand!– Learn when retreat is the better part of valor!
Learn when to change topics!
38!
8: Don’t be repetitive
Donʼt repeat the same utterance verbatim! Have different ways to say the same thing!
– Lexical choice!– Syntactic, rhetorical, and interactive structures!– Summarization/elaboration!– Speech acts/politeness!– prosody!
Donʼt ask the same query over and over!– Even when you donʼt understand!– Learn when retreat is the better part of valor!
Learn when to change topics!
See, I told you!
39!
9: Learn and customize
Take in new knowledge and vocabulary! Alignment!
– Use similar words, constructions, phrases!– Gestures, postures!
Do what works, increases engagement/satisfaction!
40!
Human Non-verbal Alignment with Virtual Human
41!
10:Maintain Immersion; don’t let the conversation Break down
Immersion: human feels engaged in the conversation, communication feels natural, enjoyable!– Focus on task!
Diagnostic: human having trouble communicating, trying to cooperatively get back to immersion!– Focus on communication!
Breakdown: human gives up, feels unable or unwilling to proceed naturally!– Focus on escape/diversion!
Martinovski & Traum 2003, “Breakdown in Human-machine interaction: the error is the clue”, in proceedings of the ISCA tutorial and research workshop on Error handling in dialogue systems, pp. 11-16.
42!
Processes to avoid or reduce Miscommunication
Before communication!– Adjust context!
Adjust interpersonal relationships! Establish agreement on signal -> meaning conventions! Introduce/focus concepts!
– Prepare partner for communication!
During communication!– Explanations, elaborations!– Monitoring & 1st turn repair!
Just after communication!– Grounding & repair!
43!
Strategies for Handling Errors
Prevent them!– Structure dialogue to simplify language of user!
E.g., “please say yes or no”!– Check correctness of understanding (verification) !
“I think you said yes, is that correct?”!
Ignore/minimize them!– Structure dialogue to partition responses at a state!– Predictions of appropriate responses!
Cope with them!– Ground Content: Acknowledge, Request repair, clarify, signal lack of or mis-
understanding! E.g. "captain i am not understanding you so well"!
– Apologize or take blame: builds social cohesion! "my English is not good captain can you repeat that again"!
– Blame user! “Stop mumbling”! ”you Americans, who can understand you”!
44!
Explicit Confirmation
45!
What to do when recognizing misunderstanding?
Ignore!– Could involve undesirable commitments!– May make communication worse!
Repair!– Could slow down conversation, make less fluent!– Could indicate more importance than merited (grounding criterion)!– Undesirable social consequences!
Loss of face for speaker: unable to communicate properly! Loss of face for addressee: unable to interpret properly!
Re-introduce!– May be difficult!– May have same consequences as Ignore & repair!
46!
Example – MRE script-follower (Rhees & Knight, unpublished, quoted in Martinovski & Traum 2003)
!1. SGT: Sir, I suggest we contact Eagle Base to request a
Medevac, but permission to secure the area first?!2. LT: Sergeant, secure the area and secure a landing
zone.!3. SGT: Yes Sir! Squad leaders! Listen up! I want 360
degree security here. First squad 12 to 4. Second squad 4 to 8. Third squad 8 to 12. Fourth squad secure the accident site. Follow our standard procedure.!
4. LT: sergeant. send one squad to recon forward on the route toward eagle one six while we wait for the medevac.!
5. Sgt: sir, should we set up a landing zone now to prepare for the medevac?!
6. LT: sergeant, I already told you to set up a landing zone for the medevac.!
7. Sgt: yes sir. sergeant duran, get your team up to the square and secure an lz. …!
8. E1-6: two six, it’s urgent you get here right now. situation’s getting critical. we’re taking fire. over.!
9. LT: eagle one six, this is eagle two six, we will be at your position as soon as possible. over.!
10. E1-6: two six, this is one six. did you read me? you’re breaking up. we need your help put here. over.!
11. LT: sergeant, send a squad to recon forward the route to eagle one six. (2nd time)!
12. E1-6: roger two six. I know you’ll get here as soon as you can. we’ll be waiting. one six out.!
13. LT: sometimes it doesn’t matter what I say.!14. Sgt: what should we do about one six if we need to
wait here for the medevac?!15. LT: sergeant, I am assuming that is a rhetorical
question.!16. Sgt: sir, we should keep our mass here to maintain
a secure area for the medevac. I suggest we send one squad to recon forward, though, to give us a secure route to one six.!
17. LT: sergeant, send one squad forward to recon for one six. (3rd time)!
18. Sgt: fourth squad. mount up.!19. Mom: what happen? you are going?!20. LT: yes, we’re leaving and your boy will probably
die.!21. E1-6: two six how much longer before we can
expect assistance? over.!22. LT: eagle one six, this is eagle two six. We’ll be
there once we finish our lattes. over.!
47!
Reacting to Disengagement from Misunderstanding
48!
Causes of breakdown
Long pauses! Over-fragmentation! Over-clarity! Repetitiveness of verbal action, syntax and
intonation! Over-coordination! Over-directedness! Lack of pragmatic adaptation! Lack of, insufficient or exaggerated state-of-mind
updates and repair requests!
49!
Signals of impending breakdown
Intonation! Ellipsis!
Hyper-Articulation! Extra-linguistics signals! Attention calls!
jokes!
50!
Summary
Companion as distinct role!– Many lessons apply to other roles!– Role models from literature!
Existing Virtual characters!– What more is needed to make them companions?!
Doʼs and Donʼts! Can we ever achieve Human-level companions?!
51!
Can we match human-level companion conversation?
52!
Questions?
53!
Schedule
