Upload
andrew-cullen
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
1Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
University of East Anglia, UK
Dicta-Sign Partner 3
Prof John Glauert
Virtual Humans Group
School of Computing Sciences
2Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Major Roles in Dicta-Sign• WP3Synthesis and Animation• Lead
• WP6Sign Language Corpora• BSL Lexicon and Corpus
• WP7 Bidirectional Integration• Lead
3Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Partnerships
4Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Virtual Signing at UEA• Dicta- Sign• John Glauert• Richard Kennaway, Post Doc• Vince Jennings, Engineer
• Face Tracking, Audio-Visual Speech • Barry Theobald
• Crowd Modelling, Urban Modelling• Prof Andy Day
5Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Independent Television
CommissionTelevirtual
University of East Anglia
The Post Office
Royal Institute for Deaf People
Instituutvoor Doven
Hamburg University
Institut für Rundfunktechnik
Institut National des Télécommunications
The ViSiCAST Consortium
6Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
The eSIGN Consortium
Televirtual
University of East Anglia
Royal National Institute for Deaf People
ViataalHamburg University
NorfolkCounty Council
7Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Sign Language Animation from Sign Gesture Notation
Prof John GlauertUniversity of East Angliawith: Norfolk County Council, RNID, Televirtual Ltd.
8Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Some Signing Avatars
9Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Virtual Characters
10Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Virtual Characters that can Sign
•
11Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Virtual Human Signing Approaches
12Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Motion Capture• Very lifelike animation
• Time-consuming to set up• Blending of signs• Combining signs from different
signers
13Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Hand-Crafted Animation• Define Key Frames• Interpolate between Key Frames
• Can give very lifelike animation
• Time consuming (maybe hours per sign)• Blending of signs still required
14Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Synthetic Signing• Synthesis from Abstract Representation
• Quick to create lexicon• a few minutes to transcribe a sign
• Instantly retargettable to any avatar with humanoid topology
• Automatic Blending
• Low Bandwidth
15Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Uses of Signing Avatars:Supporting Communication
16Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Simon the Signer, 1997-1999
• Simon-the-Signer [Broadcast TV]• Generate signed accompaniment to
broadcast• Teletext stream as source• SSE – Sign Supported English• Funded by the ITC
17Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Simon the Signer, 1997-1999
• Developing transmission technology
• virtual signer in set-top boxes• transmission of signing from text subtitles
TV Capture Card
Teletext Stream
Audio/Video Stream
Avatar Software Mixer
Computer
18Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
SiSi: Say it Sign it, 2007
• IBM with UEA and RNID• Uses Speech recogniser• Might help some meetings• Not pure BSL
• www.youtube.com/watch?v=RarMKnjqzZU
19Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
SiSi: Say it Sign it, 2007• “I’ve seen an exciting movie”
20Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
TESSA, 1998-2002
• TESSA [Retail, PO]• Using speech recognizer• Convert counter-clerk’s voice input to text• Generate sign stream from text• BSL – limited repertoire• Funded by the Post Office
21Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Post Office Counter Services
• Post Offices transact business with almost all Deaf people
• Counter Clerk asks questions using speech - No ‘back channel’ yet
• Customer• Listens or• Reads or• Watches Virtual Human Signer
• Real trials at 5 sites
22Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
VANESSA, 2002 – 2005
• Council Information Centres (CIC’s)• Signing avatar conveys signed information• Text version also displayed• Signs created from special notation• Developed and enhanced
version of TESSA (Post Office)
23Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
VANESSA – Customer’s View
24Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
VANESSA – Clerk’s View
25Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Uses of Signing AvatarsInformation Services?
26Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Web ApplicationsWeather Forecast: NGT,BSL,DGS
27Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Weather Forecasts
Today Tonight Sunday
Mostly cloudy and dry in the morning. Rain breaking out during the afternoon.Maximum temperature 13 °C.
Met Office for London and South East Met Office for London and South East EnglandEngland
Rain becoming heavy and persistent in the evening and continuing through the night.
28Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Web ApplicationsWeather Forecasts
• Grammar for normal Weather Phrases• Sign Language version for each Phrase• Forecast is sequence of Phrases• Generate Common XML Weather Model• XSLT processing for each Sign Language• XSLT processing for Spoken Languages
• Content creation by non- signers
29Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Uses of Signing AvatarsEducation?
30Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Signing Avatars in Education• PC Passport• SQA computer skills qualification• Need to be fair to all deaf candidates• Not all interpreters have IT knowledge• Different signs used by different schools• Students will have a dictionary of IT signs
31Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
PC Passport Assessment•
32Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
PC Passport Dictionary
33Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Performing Hands, 2006-2009
34Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Performing Hands, 2006-2009• GameLab London, RNID, UEA
• Teach literacy for Deaf Children• BSL and English literacy• Mixes Video, 2D Animation, Avatars• Games and Stories• Shockwave Application
• BAFTA Nominated• www.youtube.com/user/blahdeblahdeblahdebl• www.performinghands.com
35Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
LinguaSign 2006-2009
36Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
LinguaSign 2006-2009• Primary Modern Foreign Language
Learning• EU Socrates Lingua 2• Vocabulary supported by iconic gestures
(“signs”)• Speech in French, Dutch, English• Mix of conventional and synthetic
animation
37Dicta-Sign Kick Off Meeting, Athens, 06 February 2009
Thanks for your attention