Embed Size (px)
Citation preview
Does the Future Need
Translators?
Marcello Federico, FBK, Italy
Dublin, 6-7 June 2016
TAUS Industry Leaders Forum 2016
Very competitive research area
Steady performance improvements
Recent paradigm shift: from SMT to NMT
Strong adoption by translation industry
Strong adoption by global players
This naturally creates great expectations … and worries!
Progress in Machine Translation
Best MT Performance over Time @ IWSLT
0
10
20
30
40
50
60
2012 2013 2014 2015
DELTA
% EN-DE
DE-EN
RU-EN
EN-ZH
MT is a very very hard problem
and still only partially explored
Difficulty of tasks is unpredictable
we don’t know how far we are from the solution
Neural MT has improved performance
on a few languages only (so far)
State-of-the-art still bad when translating
into morphologically rich languages
between distant language pairs
Still not enough language resources available
But
Best Measured MT Performance @ IWSLT
0
10
20
30
40
50
60
70
0
5
10
15
20
25
30
35
40
45
EN-PL
EN-DE
TR-EN
DE-EN
EN-SL
ZH-EN
EN-ZH
PL-EN
AR-EN
EN-TR
FA-EN
EN-AR
RU-EN
EN-FR
RO-EN
EN-ES
ES-EN
NL-EN
EN-PT
PT-EN
Delta
BLEU
Baseline Best Delta
Examples from best WMT 2016 systems
Directio
n
MT Reference Output
TR-EN Syste
mCom
binatio
n ??
He said Lamb made the
fateful 911 call sometime
after that.
Lamb's paints 911 call, and
shortly after, he said.
CZ-EN NMT He said Lamb made the
fateful 911 call sometime
after that.
Lamb said he made a fatal phone
call to the emergency line shortly
afterwards.
RU-EN NMT+
PBMT
He said Lamb made the
fateful 911 call sometime
after that.
He reported that he made the
truthful call to 911 shortly after the
Lamb.
FI-EN PBMT He said Lamb made the
fateful 911 call sometime
after that.
He said Lamb called the dramatic
911 call for some time after that.
Translators will leverage MT more and more
get rid of routine and boring translations
MT will learn from translators
error repairs, style, terminology, …
Post-editing effort (PE) for technical translation:
< 5% PE for popular pairs EN->FR,ES,PT,IT,AR,…
< 10% PE for harder pairs EN -> RU,DE,FN,TR,ZH,JP
< 15% PE for well resources non-English
E.g. UN and EU language pairs
My optimistic 5-year predictions
Best MT Performance over Time
0
10
20
30
40
50
60
2012 2013 2014 2015
DELTA
%
EN-FR
AR-EN
ZH-EN
Skype Translator Task
Boring Translations
Creative Translations
Difficult Translation
