Dialogue Genres & Domains

How to build Tom, Dick and Harry?

CS599 - Special Topics Lecture – Shashi (Saravanan Ganesh)

One fine morning…

  Let’s make multi – domain spoken dialogue systems.

  Let’s make many virtual humans with different task goals.

  Assumptions :

  We are experts in making systems for specific domains.

  We have two pole stars:

  A Two-Stage Domain Selection Framework for Extensible Multi-Domain Spoken Dialogue Systems Mikio et. Al. Proceedings of SIGDIAL 2011

  Talking to Virtual Humans: Dialogue Models and Methodologies for Embodied Conversational Agents. David Traum 2008

  How to build systems spanning across different genres and domains?

Spoken Dialogue Interfaces

Building multi domain dialogue systems with a single interface

First Thoughts?

  Distributed: Domain Experts independently manage dialogue states and knowledge

  Independent knowledge helps in speech understanding and utterance generation

  Domains can be designed independently and added without affecting the architecture

  Domains can employ a dialogue strategy very different from other systems

  How to select an appropriate domain for each user utterance? Leader?

  Can it be used with a variety of domain experts?

  Will it be robust against speech recognition errors?

Distributed domain experts with domain selector interface Can we make this better?

Drop the leader?

  Not really. Have you heard of chaos?

  But, instead the leader now seeks information from domain experts to make a decision.

  Expert with additional two sub-modules.

  One, for estimating the probability that it is newly activated.

  One, for deciding continuation when it is already activated.

Distributed domain experts with centralized domain selector interface

What do the domain experts do?

  Use dialogue history and speech under-standing to calculate the activation probability.

  Send numerical scores, as probability to the domain selector.

  Let’s call this RECSCORE.

  But, how do we deal with erroneous shifts that may happen after every utterance?

  Let’s call this RECSCORE + BIAS

  But, the resistance to change domain must be less when a domain is likely to shift?

  Let’s call this MAXPROB

How do we get better?

  Policies!

  Current domain decides when the domain is about to shift.

  When it is decided, all remaining experts spring to life. Activation Probability.

  Let’s call this NOACTIVPROB.

  Current domains also flags out-of-domain (OOD) utterances.

  A variation used Activation Probability at every utterance to help the preceding domain expert.

  Let’s call this FULLIMPL.

Distributed domain experts with centralized domain selector interface

Answers.

SYSTEM WEIGHTED AVERAGE F1 SCORES

RECSCORE 0.789

RECSCORE + BIAS 0.838

MAXPROB 0.832

NOACTIVPROB 0.849

FULLIMPL 0.883

Virtual Human Systems

Building explicit computational models of behavior to emulate different people

First Thoughts?

  Beyond the start of art to build virtual humans with same capabilities as real people.

  Fidelity in modelling human behavior, more important than effectiveness.

  Cyclical approach, with multiple passes. A full system early in the process.

  Explicit computational models of behavior for each virtual human.

  Start with ‘observing’ people in ‘similar activities’ that we have to emulate to build a computational model and ‘analyze’ the model by again observing its use.

How to observe?

  Observation provide insight on some of the important aspects of behaviors.

  These aspects help us build what are called theories.

  There are now two ways to build computational models, from these observations.

  1) Convert theories into rules or 2) Convert them into features.

  Thus, building a virtual human requires enumerate skills repeated in a cyclic approach.

How to choose theories?

  Represent Phenomenon if there is general evidence of its presence in a cognitive model in some domains.

  Represent phenomenon only if there is evidence from data that it occurs in this domain.

  Represent phenomenon only if it leads to a functional consequence in agent behavior.

  Represent phenomenon only if it is the simplest way to achieve the consequence

  Represent phenomenon only if it leads to a necessary function for the domain tasks the character must perform.

What are similar activities?

  Procedures: type, purpose, function.

  Roles: competence, obligation, rights.

  Instruments: machines, media

  Other physical environment.

  Number and nature of the participants and the activities.

Information State Approach

  Dialogues as static part, consisting of a set of information state components and current values.

  Dialogue acts as dynamic parts that change the information state components.

  Interpretation : Produce a hypothesis about dialogue acts that have been performed.

  Update : Change the information state in accordance to the performance.

  Selection : Deciding what to do give the current information state.

  Realization : Deciding on an ordered set of physical behavior to perform the selected dialogue acts.

Question Answering Characters

  Characters that have a set of knowledge that they can impart when asked, and goals for the presentation of this information.

  Specimen : Sgt. Blackwell, can be interviewed about ICT, army and virtual humans.

  Information State: Last few utterances, two threshold values to avoid repetition.

  Translation Model to map a language model for questions to answers.

Group Conversation Characters

  Characters that server as background characters for large virtual simulations. Their behavior should be natural for a crows, engaged in conversational interactions.

  Information State: Set of characters, and conversations. Each conversation has a set of participants, a turn-holder, a transition relevance place (TRP), and sequences of utterances.

Advanced Virtual Humans

  Virtual humans that engage in multiparty teamwork and non-term negotiation.

  Information State : Richer model, to accommodate for information to accept and select diverse dialogue acts.

Thank You!

Time to go out and build our favorite multi-talented robots.