ICANN workshop, September 14,Athens, Greece

Intelligent Affective Interaction

technologies and applications

workshop focus

• present/discuss experiences from FP6 projects in the field of multimodal interfaces, as well as related fields (robotics, e-inclusion, cognition, etc.)

• discuss the position of the European Research Area within the state of the art

• how this position can be maintained and improved within FP7

workshop presentations

• Prof. C. Pelachaud, Univ. Paris 8, FR• Prof. Ar. Pnevmatikakis, AIT, GR• Dr. V. Moschou, AUTH, GR• Dr. K. Moustakas, ITI-CERTH, GR• Dr. D. Arnone, Engineering, IT• Dr. M. Price, BBC R&D, UK• Dr. K. Karpouzis, ICCS/NTUA, GR• Organized by Prof. Stefanos Kollias,

ICCS/NTUA, GR

lessons learned from HUMAINE

… and the way forward

4

Kostas Karpouzis, ICCS-NTUA, Greece

so, what is HUMAINE?

• HUMAINE stands for “human-machine interaction network on emotions”

• EU FP6 Network of Excellence– IST thematic priority: Multimodal

Interfaces– EC financial contribution: 4.95M€– duration: 48 months (01/04-12/07)

so, what is HUMAINE?

• 33 partner institutions from many disciplines– computer

science, psychology, human factors, SMEs, …

why HUMAINE?

• the FP6 work program indicates emotion as one of the key factors of natural HCI

• but, what is emotion?– engineers do not touch this question– psychologists do, but need to express answers

with measurable, quantitative terms– since it’s an IST project, we need to find a way

to put concepts and findings to actual use (IPs)

• the multi-disciplinary nature of HUMAINE shows that all are needed!

• theories and models of emotion• signals to signs of emotion• data and databases• emotion in interaction• emotion in cognition & action• emotion in communication &

persuasion• usability of emotion-oriented systems• ethics and good practice

thematic areas

HUMAINE main deliverables

• as described in the original Technical Annex

• a community of researchers/developers/users/…

• joint awareness of suitable methods• scientific basis for making research

cumulative

the HUMAINE Research Area

HUMAINE in the state of the art

• leading European academia in the fields of emotion representation, ECAs and autonomous robots

• successful groups in the fields of innovative multimodal signal analysis and understanding

• leaders in naturalistic data capture and annotation

• application-related groups and SMEs– design, evaluate, test, use

the real HUMAINE deliverables

• scientific community– bringing together teams from diverse

background in joint projects (exemplars)

• Europe still has the lead in emotion-related research– mainly thanks to FP5 and FP6– but others (US, Japan) are catching up fast!

• the leading portal on emotion-research (http://emotion-research.net)

the real HUMAINE deliverables

• again, what is emotion?– in the context of everyday HCI – and in the context of IST

• bits and bytes in related fields– analysis and synthesis tools (visual,

aural, physiological, etc.)– specific applications (call centers,

cocktail party, helping the disabled and elderly)

– deployed now

possible related applications

• ECAs driven by outside, text sources (e.g. weather reports or traffic announcements) to produce sign language for the deaf

• immersion in virtual or augmented environments for interaction entertainment

• autonomous machines (robots) that learn and adapt to user traits and assist everyday life

how to keep the advantage

• HUMAINE is an IST network of excellence– contradiction in terms?– IST is about applied technologies– NoEs are research incubators

• ethics is a major issue– hence, all steps need to be careful– NoEs are the perfect place to build

prototypes…– …to be polished and deployed by

Integrated Projects (IPs)

how to keep the advantage

• putting technological excellence into practice

• IPs in the ‘multimodal interfaces’ thematic priority need to build on the concepts provided by the NoEs

• the CALLAS IP is one of these outlets– facial and speech analysis, emotion

recognition– augmented reality, ECAs– adaptation, personalization

how to keep the advantage

• FP7 draft work program uses terms like ‘multimodal’, ‘emotion’, ‘affect’ in a number of diverse concepts– interfaces, robotics, cognition, assisted

living• careful about using the word ‘emotion’

in applications– sometimes tends to move focus on

representation and ethics, instead of interfaces and HCI

– small concepts from research areas ready to use in applications