Upload
bathsheba-jenkins
View
216
Download
1
Tags:
Embed Size (px)
Citation preview
gUI: Specifying Complete User gUI: Specifying Complete User InteractionInteraction
Soft computing LaboratoryYonsei University
October 25, 2004
2
ContentsContents• Computer human user interfaces – Synthesis• Value of empathy in synthesis• Computer human user interfaces – Analysis• gUI research objectives• The role of the anthropomorphic interface metaphor• gUI data specification issues• Implementation issues• Results of the prototype system• Conclusion
3
Computer Human User Interfaces – SynthesisComputer Human User Interfaces – Synthesis
• Humane CHIs (Computer Human Interfaces)– Embodied character agents (ECAs), Embodied
conversational agents, virtual conversational characters, virtual humans, talking heads
• Basic criteria for a standard markup language– Completeness, simplicity, consistency, intuitiveness,
abstraction, usability, standardization, evaluation
• VHML– An open specification available at the VHML web site– Two applications
MetaFace, an ECAThe Mentor systems, a dialogue management system
4
Value of Empathy in SynthesisValue of Empathy in Synthesis• ECA with emotions or a personality: Increase the feeling of
familiarity
• Application areas with common anthropomorphic metaphors– Universities– E-commerce– Web guides– Traveling– Interactive games– Knowledge interactive companions
5
Computer Human User Interfaces – AnalysisComputer Human User Interfaces – Analysis
• “Aware”: The user does not know how to perform the task
• Gestalt user interfaces (gUIs)– “gestalt”: An organized whole in which each part affects
every other part– The mind organizes events and situations as a pattern “big
picture”– The entities
Similarity, proximity, closure, continuity, membership
– Input: A combined “stimulus” of text, button clicks, and analyzed facial, vocal, and emotional gestures
– Recognition: Translate the stimulus into the intent
6
Value of Empathy in AnalysisValue of Empathy in Analysis• “more complete” encapsulation of human interaction
– The empathic ability of an interface
– The extra functionality acquired
– An intuitive interface
• Overlooked empathic state of the virtual character– Attend: Considering the thoughts and feelings of the user
– Engage: Aligning actions, thoughts, and feelings with those of the user
– Value: Expressing the value of the user’s interaction
– Encourage: Expression encouragement for further interaction
– Parting: Suspending dialogue while user performs other tasks
– Available: Allowing the user to interrupt for interaction
gUIs
7
gUI Research ObjectivesgUI Research Objectives• Formal specification, development, implementation,
evaluation, and standardization of a markup language is necessary to provide a stable, consistent base for both industrial use and future research into multi-modal Human-Computer interaction
• MPEG-4: Solutions to facial and body animation at the low level
• Require a unified standard architecture plus language to control/record the higher level human ECA interaction
8
The Role of the Anthropomorphic Interface The Role of the Anthropomorphic Interface Metaphor (1)Metaphor (1)
• Anthropomorphic metaphor– To specify complete user interaction with an ECA– To use the analysis and synthesis of emotion and gesture
• Desktop metaphor– The most predominant metaphor used in computer systems– The metaphor’s likeness to our perception of the world
• “illusions”– Reduce cognitive overhead when users interact with an
anthropomorphic interface– gUI applications become an extension of real life
Waste paper basket on the Macintosh desktopThe humanity of talking heads
9
The Role of the Anthropomorphic Interface The Role of the Anthropomorphic Interface Metaphor (2)Metaphor (2)
• Anthropomorphic web applications– Bryan and Gerchman
Browsing: The leisurely navigation along an arbitrary path Searching: Goal oriented finding specific information
– A knowledgable ECA (MetaFace) Knows that you are getting frustrated in web searching Offers assistance as a friend
• Transcription– Representation, transformation, and storage of analyzed emotions
and gestures
• Real-time interaction and data representation– Efficient storage and communication of the data– Intelligent television application: handle real-time emotions and
gestures to control the function of the TV
10
gUI Data Specification Issues (1)gUI Data Specification Issues (1)• “Affect recognition”
– Proper gestalt understanding of the user requirements– “Non-verbal” behavior and “Non-verbal” communications
web site (http://www3.usal.es/~nonverbal/paper.htm)
• Temporal window– Differentiate short mixed emotions such as surprise before
anger or fear– Detect more lengthy emotions such as gradual happiness as
“understanding comes to someone”
• Specification of the gUI input data format– At the lowest level: Efficient binary format– At the higher levels: a more structured format aiding in
abstraction and cognitive understanding
11
gUI Data Specification Issues (2)gUI Data Specification Issues (2)
12
gUI Data Specification Issues (3)gUI Data Specification Issues (3)• Dynamic context transitions
• Time-stamping of emotion and gestures: Synchronizing the resulting XML document and providing context
• MPEG-4 face and body animation– Efficient communication standard– Emotion and gesture representation for real-time
applications
• Some problems in VHML data representation– Lack of a consistent set of emotions, and the ability to add
more– Lack of ability to show change in different sets of emotions
or just a single emotion– Lack of additional attributes for emotion besides intensity
13
gUI Data Specification Issues (4)gUI Data Specification Issues (4)• A more realistic and extensible semantic structure
• Gesture input
14
gUI Data Specification Issues (5)gUI Data Specification Issues (5)• Issues on specifying gestures
– A huge diversity of gestures for the human body to try and identify a base set for all analysis and synthesis applications
– Own associated information on each gesture: No standard parameters for all gestures
– Culturally dependent gestures
• A less naïve format
15
Implementation Issues (1)Implementation Issues (1)• MetaFace framework
– An example ECA used to test the underlying theories– Based on MPEG-4 and VHML– The ability to vocalize and visually display emotion and
gesture– Dialogue management tool language (DMTL)
• Evaluate tag: Inference the context
16
Implementation Issues (2)Implementation Issues (2)
17
ConclusionConclusion• Challenges
– The overlap of text input, gesture, and emotion as one coherent stimulus
– Standardization of emotion and gesture representation
• Complete user interaction with gestalt user interface– Use anthropomorphic metaphors– Reduce the conceptual load– Mentor and MetaFace research project