MT and Translator's Tools

Preview:

Citation preview

OutlineTools for TranslatorsFree Language DataMachine Translation

Machine Translation and Translation Technology

Jimmy O’Regan

The Apertium Project

OSS Bar Camp, 19 September 2009

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

1 Free Language Data

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Tools for Translators

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Some Terminology

InternationalisationGiving software the capability to display text in anotherlanguage

In Open Source, this generally means adding support forgettext.

LocalisationCustomising the messages displayed to the user to appear inthe manner most appropriate for them.In their language, or their dialect.

TranslationConverting text from one language to another

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Some Terminology

InternationalisationGiving software the capability to display text in anotherlanguageIn Open Source, this generally means adding support forgettext.

LocalisationCustomising the messages displayed to the user to appear inthe manner most appropriate for them.In their language, or their dialect.

TranslationConverting text from one language to another

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Some Terminology

InternationalisationGiving software the capability to display text in anotherlanguageIn Open Source, this generally means adding support forgettext.

Localisation

Customising the messages displayed to the user to appear inthe manner most appropriate for them.In their language, or their dialect.

TranslationConverting text from one language to another

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Some Terminology

InternationalisationGiving software the capability to display text in anotherlanguageIn Open Source, this generally means adding support forgettext.

LocalisationCustomising the messages displayed to the user to appear inthe manner most appropriate for them.

In their language, or their dialect.

TranslationConverting text from one language to another

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Some Terminology

InternationalisationGiving software the capability to display text in anotherlanguageIn Open Source, this generally means adding support forgettext.

LocalisationCustomising the messages displayed to the user to appear inthe manner most appropriate for them.In their language, or their dialect.

TranslationConverting text from one language to another

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Some Terminology

InternationalisationGiving software the capability to display text in anotherlanguageIn Open Source, this generally means adding support forgettext.

LocalisationCustomising the messages displayed to the user to appear inthe manner most appropriate for them.In their language, or their dialect.

Translation

Converting text from one language to another

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Some Terminology

InternationalisationGiving software the capability to display text in anotherlanguageIn Open Source, this generally means adding support forgettext.

LocalisationCustomising the messages displayed to the user to appear inthe manner most appropriate for them.In their language, or their dialect.

TranslationConverting text from one language to another

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localisation and translation are sometimes, but not always, thesame.

Documents may need to be localised, but not translated:A British company with an Irish office still needs to localise their

documents: any reference to “our London office” will need to bechanged to “our Dublin office”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localisation and translation are sometimes, but not always, thesame.Documents may need to be localised, but not translated:

A British company with an Irish office still needs to localise theirdocuments: any reference to “our London office” will need to bechanged to “our Dublin office”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localisation and translation are sometimes, but not always, thesame.Documents may need to be localised, but not translated:A British company with an Irish office still needs to localise their

documents: any reference to “our London office” will need to bechanged to “our Dublin office”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localised translations can also have additional requirements:

gettext allows numbers to be specially treated: ‘‘%dfile(s)’’ ugliness is not necessary.English and Spanish need two forms of words for number: single

and pluralPolish needs three: single, plural, and quantity (greater than 5)Slovenian needs four: single, dual, plural, and quantity.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localised translations can also have additional requirements:gettext allows numbers to be specially treated: ‘‘%dfile(s)’’ ugliness is not necessary.

English and Spanish need two forms of words for number: singleand pluralPolish needs three: single, plural, and quantity (greater than 5)Slovenian needs four: single, dual, plural, and quantity.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localised translations can also have additional requirements:gettext allows numbers to be specially treated: ‘‘%dfile(s)’’ ugliness is not necessary.English and Spanish need two forms of words for number

: singleand pluralPolish needs three: single, plural, and quantity (greater than 5)Slovenian needs four: single, dual, plural, and quantity.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localised translations can also have additional requirements:gettext allows numbers to be specially treated: ‘‘%dfile(s)’’ ugliness is not necessary.English and Spanish need two forms of words for number: single

and plural

Polish needs three: single, plural, and quantity (greater than 5)Slovenian needs four: single, dual, plural, and quantity.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localised translations can also have additional requirements:gettext allows numbers to be specially treated: ‘‘%dfile(s)’’ ugliness is not necessary.English and Spanish need two forms of words for number: single

and pluralPolish needs three

: single, plural, and quantity (greater than 5)Slovenian needs four: single, dual, plural, and quantity.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localised translations can also have additional requirements:gettext allows numbers to be specially treated: ‘‘%dfile(s)’’ ugliness is not necessary.English and Spanish need two forms of words for number: single

and pluralPolish needs three: single, plural, and quantity (greater than 5)

Slovenian needs four: single, dual, plural, and quantity.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localised translations can also have additional requirements:gettext allows numbers to be specially treated: ‘‘%dfile(s)’’ ugliness is not necessary.English and Spanish need two forms of words for number: single

and pluralPolish needs three: single, plural, and quantity (greater than 5)Slovenian needs four

: single, dual, plural, and quantity.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation vs. Translation

Localised translations can also have additional requirements:gettext allows numbers to be specially treated: ‘‘%dfile(s)’’ ugliness is not necessary.English and Spanish need two forms of words for number: single

and pluralPolish needs three: single, plural, and quantity (greater than 5)Slovenian needs four: single, dual, plural, and quantity.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation

Software localisation is a huge business area for proprietarysoftware.

One that traditionally lags behind Open Source.That advantage is usually due to the efforts of a handful ofdedicated volunteers for the majority of languages.But they’re catching up: Facebook is using Open Source-likeefforts for their translations.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation

Software localisation is a huge business area for proprietarysoftware.One that traditionally lags behind Open Source.

That advantage is usually due to the efforts of a handful ofdedicated volunteers for the majority of languages.But they’re catching up: Facebook is using Open Source-likeefforts for their translations.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation

Software localisation is a huge business area for proprietarysoftware.One that traditionally lags behind Open Source.That advantage is usually due to the efforts of a handful ofdedicated volunteers for the majority of languages.

But they’re catching up: Facebook is using Open Source-likeefforts for their translations.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation

Software localisation is a huge business area for proprietarysoftware.One that traditionally lags behind Open Source.That advantage is usually due to the efforts of a handful ofdedicated volunteers for the majority of languages.But they’re catching up: Facebook is using Open Source-likeefforts for their translations.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Localisation Tools

Unsurprisingly then, localisation is very well supported by OpenSource software:

Pootle (http://translate.sourceforge.net/wiki/pootle/index -Web-based)

Virtaal (http://translate.sourceforge.net/wiki/virtaal/index -cross platform)

poEdit (http://www.poedit.net/ - cross platform)

Lokalize (http://userbase.kde.org/Lokalize - KDE)

GTranslator (http://gtranslator.sourceforge.net/ - GNOME)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Translation Tools

Unfortunately, there’s only one real equivalent tool for generaltranslation: OmegaT

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Common Features

All of these tools include these features:

Translation MemoryAutomatically reuse previous translations

Fuzzy matchingSuggest translations similar to previously translated sentences

Terminology ManagementGive suggestions from a per-project dictionary

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Common Features

All of these tools include these features:

Translation Memory

Automatically reuse previous translations

Fuzzy matchingSuggest translations similar to previously translated sentences

Terminology ManagementGive suggestions from a per-project dictionary

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Common Features

All of these tools include these features:

Translation MemoryAutomatically reuse previous translations

Fuzzy matchingSuggest translations similar to previously translated sentences

Terminology ManagementGive suggestions from a per-project dictionary

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Common Features

All of these tools include these features:

Translation MemoryAutomatically reuse previous translations

Fuzzy matching

Suggest translations similar to previously translated sentences

Terminology ManagementGive suggestions from a per-project dictionary

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Common Features

All of these tools include these features:

Translation MemoryAutomatically reuse previous translations

Fuzzy matchingSuggest translations similar to previously translated sentences

Terminology ManagementGive suggestions from a per-project dictionary

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Common Features

All of these tools include these features:

Translation MemoryAutomatically reuse previous translations

Fuzzy matchingSuggest translations similar to previously translated sentences

Terminology Management

Give suggestions from a per-project dictionary

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Common Features

All of these tools include these features:

Translation MemoryAutomatically reuse previous translations

Fuzzy matchingSuggest translations similar to previously translated sentences

Terminology ManagementGive suggestions from a per-project dictionary

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

TerminologyLocalisation vs. Translation

Translate Toolkit

http://translate.sourceforge.net/A set of common tools for translation/localisation:

Translation Memory server

Format conversion

Terminology management

Quality control tools

All brought to you by the wonderful people of translate.org.za

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Free Language Software Needs Free Data

Just like “Free Software Needs Free Documentation”, so too doesFree Language Software need Free Data.

Usually, this means we have to make it ourselves.Unfortunately, the community of developers of free languagesoftware, and thus free language data, is quite small.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Free Language Software Needs Free Data

Just like “Free Software Needs Free Documentation”, so too doesFree Language Software need Free Data.Usually, this means we have to make it ourselves.

Unfortunately, the community of developers of free languagesoftware, and thus free language data, is quite small.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Free Language Software Needs Free Data

Just like “Free Software Needs Free Documentation”, so too doesFree Language Software need Free Data.Usually, this means we have to make it ourselves.Unfortunately, the community of developers of free languagesoftware, and thus free language data, is quite small.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Spell checking data packages are the absolute bare minimum ofsupport for a language with technology.Usually, the people who develop them tend to be involved in otherareas of Free language software:

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Kevin Scannell

Makes the Irish spell checking data. And An Gramadoir, an Irishlanguage grammar checker. And created a WordNet/thesaurus forIrish. And contributed the language data for Apertium’sIrish–Scots Gaelic translator. etc.Marcin Mi lkowskiHeavily involved in the Polish spell checking data AndLanguageTool, a multilingual grammar checker that’s integratedwith Open Office. And maintains the open Polish thesaurus.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Kevin ScannellMakes the Irish spell checking data.

And An Gramadoir, an Irishlanguage grammar checker. And created a WordNet/thesaurus forIrish. And contributed the language data for Apertium’sIrish–Scots Gaelic translator. etc.Marcin Mi lkowskiHeavily involved in the Polish spell checking data AndLanguageTool, a multilingual grammar checker that’s integratedwith Open Office. And maintains the open Polish thesaurus.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Kevin ScannellMakes the Irish spell checking data. And An Gramadoir, an Irishlanguage grammar checker.

And created a WordNet/thesaurus forIrish. And contributed the language data for Apertium’sIrish–Scots Gaelic translator. etc.Marcin Mi lkowskiHeavily involved in the Polish spell checking data AndLanguageTool, a multilingual grammar checker that’s integratedwith Open Office. And maintains the open Polish thesaurus.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Kevin ScannellMakes the Irish spell checking data. And An Gramadoir, an Irishlanguage grammar checker. And created a WordNet/thesaurus forIrish.

And contributed the language data for Apertium’sIrish–Scots Gaelic translator. etc.Marcin Mi lkowskiHeavily involved in the Polish spell checking data AndLanguageTool, a multilingual grammar checker that’s integratedwith Open Office. And maintains the open Polish thesaurus.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Kevin ScannellMakes the Irish spell checking data. And An Gramadoir, an Irishlanguage grammar checker. And created a WordNet/thesaurus forIrish. And contributed the language data for Apertium’sIrish–Scots Gaelic translator. etc.

Marcin Mi lkowskiHeavily involved in the Polish spell checking data AndLanguageTool, a multilingual grammar checker that’s integratedwith Open Office. And maintains the open Polish thesaurus.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Kevin ScannellMakes the Irish spell checking data. And An Gramadoir, an Irishlanguage grammar checker. And created a WordNet/thesaurus forIrish. And contributed the language data for Apertium’sIrish–Scots Gaelic translator. etc.Marcin Mi lkowski

Heavily involved in the Polish spell checking data AndLanguageTool, a multilingual grammar checker that’s integratedwith Open Office. And maintains the open Polish thesaurus.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Kevin ScannellMakes the Irish spell checking data. And An Gramadoir, an Irishlanguage grammar checker. And created a WordNet/thesaurus forIrish. And contributed the language data for Apertium’sIrish–Scots Gaelic translator. etc.Marcin Mi lkowskiHeavily involved in the Polish spell checking data

AndLanguageTool, a multilingual grammar checker that’s integratedwith Open Office. And maintains the open Polish thesaurus.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Kevin ScannellMakes the Irish spell checking data. And An Gramadoir, an Irishlanguage grammar checker. And created a WordNet/thesaurus forIrish. And contributed the language data for Apertium’sIrish–Scots Gaelic translator. etc.Marcin Mi lkowskiHeavily involved in the Polish spell checking data AndLanguageTool, a multilingual grammar checker that’s integratedwith Open Office.

And maintains the open Polish thesaurus.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Kevin ScannellMakes the Irish spell checking data. And An Gramadoir, an Irishlanguage grammar checker. And created a WordNet/thesaurus forIrish. And contributed the language data for Apertium’sIrish–Scots Gaelic translator. etc.Marcin Mi lkowskiHeavily involved in the Polish spell checking data AndLanguageTool, a multilingual grammar checker that’s integratedwith Open Office. And maintains the open Polish thesaurus.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

translate.org.zaMake the spelling checkers for several South African languages, aswell as many tools for translators already mentioned–Virtaal,Translate Toolkit, Pootle. Much of Apertium’s English–Africanstranslator was made directly by translate.org.za developers, as wellas Apertium’s dbus interface, and a GUI. (Virtaal allows translatorsto use machine translations as a basis for their work).

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

The importance of spell checkers

Spell checkers are set to become even more important. Hunspell,which is fast becoming the standard spell checker in Open Sourceprojects, now includes morphological analysis and generation. Thiswill greatly improve, among other things, terminology managementin translator’s tools.At the moment, if you have “dog” in your terminology list, thetranslation tool will see that and only that: “dogs” will gounrecognised. With morphological analysis, the tool can know that“dogs” is not only related to “dog”, but is the plural of a noun:another assistance to the translator.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Machine Translation

Machine translation has a bad reputation.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Mechanical Translation

Mechanical translation, of any form, does tend, inevitably, to havemistakes.

For centuries, paintings of Moses portrayed him as having horns.A translator of the Latin Vulgate added the wrong vowel: hethought that Moses had horns, not that his face was glowing.And people were killed for wishing to correct that, and othermistakes.Proofread translations, always.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Mechanical Translation

Mechanical translation, of any form, does tend, inevitably, to havemistakes.For centuries, paintings of Moses portrayed him as having horns.

A translator of the Latin Vulgate added the wrong vowel: hethought that Moses had horns, not that his face was glowing.And people were killed for wishing to correct that, and othermistakes.Proofread translations, always.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Mechanical Translation

Mechanical translation, of any form, does tend, inevitably, to havemistakes.For centuries, paintings of Moses portrayed him as having horns.A translator of the Latin Vulgate added the wrong vowel: hethought that Moses had horns, not that his face was glowing.

And people were killed for wishing to correct that, and othermistakes.Proofread translations, always.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Mechanical Translation

Mechanical translation, of any form, does tend, inevitably, to havemistakes.For centuries, paintings of Moses portrayed him as having horns.A translator of the Latin Vulgate added the wrong vowel: hethought that Moses had horns, not that his face was glowing.And people were killed for wishing to correct that, and othermistakes.

Proofread translations, always.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Mechanical Translation

Mechanical translation, of any form, does tend, inevitably, to havemistakes.For centuries, paintings of Moses portrayed him as having horns.A translator of the Latin Vulgate added the wrong vowel: hethought that Moses had horns, not that his face was glowing.And people were killed for wishing to correct that, and othermistakes.Proofread translations, always.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup

– the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory

– also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation

– considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation

– currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate.

Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one.

And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation

– The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s.

– The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best

!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Types of Machine Translation

Dictionary lookup – the most basic form of MT

Translation Memory – also, a basic form of MT

Example Based Machine Translation – considered the mostaccurate form of MT, but there are few if any examples “inthe wild”.

Statistical Machine Translation – currently the darling ofresearch and the basis of Google Translate. Solves a lot of oldproblems, but introduces new one. And breaks a lot of thingsthat “used to work”.

Rule Based Machine Translation – The oldest kind of MT,dating back to the 1950s. – The kind I work with, soobviously it’s the best!!!

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Is Machine Translation a Translator’s Tool?

Yes.

That might be hard to accept. Particularly if you only speakEnglish. But for closely-related, similar languages, machinetranslation can be as effective and accurate as a spelling checker.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Is Machine Translation a Translator’s Tool?

Yes.That might be hard to accept.

Particularly if you only speakEnglish. But for closely-related, similar languages, machinetranslation can be as effective and accurate as a spelling checker.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Is Machine Translation a Translator’s Tool?

Yes.That might be hard to accept. Particularly if you only speakEnglish.

But for closely-related, similar languages, machinetranslation can be as effective and accurate as a spelling checker.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Is Machine Translation a Translator’s Tool?

Yes.That might be hard to accept. Particularly if you only speakEnglish. But for closely-related, similar languages, machinetranslation can be as effective and accurate as a spelling checker.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Uses of Machine Translation

1 Assimilation Understanding a text

2 Dissemination Preparing a text for translation. That is; forpreparing a rough draft for a translator. Who then edits thetext.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Uses of Machine Translation

1 Assimilation

Understanding a text

2 Dissemination Preparing a text for translation. That is; forpreparing a rough draft for a translator. Who then edits thetext.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Uses of Machine Translation

1 Assimilation Understanding a text

2 Dissemination Preparing a text for translation. That is; forpreparing a rough draft for a translator. Who then edits thetext.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Uses of Machine Translation

1 Assimilation Understanding a text

2 Dissemination

Preparing a text for translation. That is; forpreparing a rough draft for a translator. Who then edits thetext.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Uses of Machine Translation

1 Assimilation Understanding a text

2 Dissemination Preparing a text for translation.

That is; forpreparing a rough draft for a translator. Who then edits thetext.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Uses of Machine Translation

1 Assimilation Understanding a text

2 Dissemination Preparing a text for translation. That is; forpreparing a rough draft for a translator. Who then edits thetext.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

“Phrase-Based” SMT

Most current research (and commercial use) of Statistical MT uses“phrase-based” SMT.

The problem is it’s not phrase-based.It’s N-Gram based.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

“Phrase-Based” SMT

Most current research (and commercial use) of Statistical MT uses“phrase-based” SMT.The problem is it’s not phrase-based.

It’s N-Gram based.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

“Phrase-Based” SMT

Most current research (and commercial use) of Statistical MT uses“phrase-based” SMT.The problem is it’s not phrase-based.It’s N-Gram based.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Grams

An n-gram is a collection of “n” amounts of tokens

For text, these are usually (not always!) words...punctuation is counted as a “word”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Grams

An n-gram is a collection of “n” amounts of tokensFor text, these are usually (not always!) words

...punctuation is counted as a “word”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Grams

An n-gram is a collection of “n” amounts of tokensFor text, these are usually (not always!) words...punctuation is counted as a “word”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Unigrams

“This is John’s dog.”

Example

ThisisJohn’sdog.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Unigrams

“This is John’s dog.”

Example

ThisisJohn’sdog.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Bigrams

“This is John’s dog.”

Example

This isis JohnJohn ’s’s dogdog .

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Bigrams

“This is John’s dog.”

Example

This isis JohnJohn ’s’s dogdog .

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Trigrams“This is John’s dog.”

Example

This is Johnis John ’sJohn ’s dog’s dog .

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Trigrams“This is John’s dog.”

Example

This is Johnis John ’sJohn ’s dog’s dog .

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

An n-gram language model is a collection of n-grams

for n..1: atrigram model includes trigrams, bigrams, and unigrams.Each n-gram is listed along with its frequency(According to a particular corpus)But, most importantly...

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

An n-gram language model is a collection of n-grams for n..1: atrigram model includes trigrams, bigrams, and unigrams.

Each n-gram is listed along with its frequency(According to a particular corpus)But, most importantly...

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

An n-gram language model is a collection of n-grams for n..1: atrigram model includes trigrams, bigrams, and unigrams.Each n-gram is listed along with its frequency

(According to a particular corpus)But, most importantly...

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

An n-gram language model is a collection of n-grams for n..1: atrigram model includes trigrams, bigrams, and unigrams.Each n-gram is listed along with its frequency(According to a particular corpus)

But, most importantly...

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

An n-gram language model is a collection of n-grams for n..1: atrigram model includes trigrams, bigrams, and unigrams.Each n-gram is listed along with its frequency(According to a particular corpus)But, most importantly...

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

N-grams overlap.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

When a sequence of words is queried against a language model,the language model software computes the combined likelihood of1..n combinations in that sequence.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

Possibly the first use of n-gram language models was in AutomaticSpeech Recognition.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

For basic uses of ASR, such as call centres, a custom grammar isused.

In a mobile phone, such a grammar could look like this:

Example

CALLWORD : phone call dialZEROWORD : zero ohNUMBER : one two three four five six seven eight nineZEROWORDNUMBERS : NUMBER* NUMBERCOMMAND : CALLWORD NUMBERS

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

For basic uses of ASR, such as call centres, a custom grammar isused.In a mobile phone, such a grammar could look like this:

Example

CALLWORD : phone call dialZEROWORD : zero ohNUMBER : one two three four five six seven eight nineZEROWORDNUMBERS : NUMBER* NUMBERCOMMAND : CALLWORD NUMBERS

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

However, for continuous dictation, building such grammars is analmost infinite task.

Instead of defining long, complicated grammars that define, forexample, when the sound /mi:t/ represents “meet” and when itrepresents “meat”, n-gram language models allow the correctsound to be chosen based on the context of the surrounding words.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

However, for continuous dictation, building such grammars is analmost infinite task.Instead of defining long, complicated grammars that define, forexample, when the sound /mi:t/ represents “meet” and when itrepresents “meat”, n-gram language models allow the correctsound to be chosen based on the context of the surrounding words.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

This was an obvious progression for ASR, which uses statisticalmodelling to choose in context which sound is most likely, basedon the surrounding sounds.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

Now, language models are being used in all areas of languagetechnology.The problem is: useful language models are huge, and can becomputationally costly to use

unless you have a data centre.Google, for example, use language models for everything :

Spell checking (Search, Google Wave, GMail)

Search queries (“Did you mean?”)

Machine translation

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

Now, language models are being used in all areas of languagetechnology.The problem is: useful language models are huge, and can becomputationally costly to use unless you have a data centre.

Google, for example, use language models for everything :

Spell checking (Search, Google Wave, GMail)

Search queries (“Did you mean?”)

Machine translation

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram Language Models

Now, language models are being used in all areas of languagetechnology.The problem is: useful language models are huge, and can becomputationally costly to use unless you have a data centre.Google, for example, use language models for everything :

Spell checking (Search, Google Wave, GMail)

Search queries (“Did you mean?”)

Machine translation

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram-Based SMT

Basic Statistical MT uses a probabilistic dictionary: each word pairhas a probability assigned.

The interesting part is how they get those dictionaries.A program, usually GIZA++ (Open Source), reads two pairs oftext: the source language, and the target language.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram-Based SMT

Basic Statistical MT uses a probabilistic dictionary: each word pairhas a probability assigned.The interesting part is how they get those dictionaries.

A program, usually GIZA++ (Open Source), reads two pairs oftext: the source language, and the target language.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram-Based SMT

Basic Statistical MT uses a probabilistic dictionary: each word pairhas a probability assigned.The interesting part is how they get those dictionaries.A program, usually GIZA++ (Open Source), reads two pairs oftext: the source language, and the target language.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram-Based SMT

For each word in each sentence of the source language, theprobable translation is considered to be every word in the target; asmore words are seen, the translations are re-evaluated: the nexttime word 1 is used, perhaps “possible translation 1” is present,but “possible translation 2” is absent from the sentence: theprobability of the former is increased; the latter, decreased.And so on, over the course of the text, the probabilities of eachword are re-evaluated; then the whole text is processed again, andagain, until a reasonable level of probability remains.The result is a probabilistic dictionary.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram-Based SMT

N-grams come into the picture in two ways:

1 Evaluating multiple probable translationsSimilarly to Speech Recognition, each choice is evaluatedagainst a language model

2 N-grams as “words”As well as considering individual words, each n-gram isconsidered as a possible “phrase”, and treated as an individualword. This helps to cut down on ambiguous terms:“basketball coach” vs. “coach driver”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram-Based SMT

N-grams come into the picture in two ways:

1 Evaluating multiple probable translations

Similarly to Speech Recognition, each choice is evaluatedagainst a language model

2 N-grams as “words”As well as considering individual words, each n-gram isconsidered as a possible “phrase”, and treated as an individualword. This helps to cut down on ambiguous terms:“basketball coach” vs. “coach driver”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram-Based SMT

N-grams come into the picture in two ways:

1 Evaluating multiple probable translationsSimilarly to Speech Recognition, each choice is evaluatedagainst a language model

2 N-grams as “words”As well as considering individual words, each n-gram isconsidered as a possible “phrase”, and treated as an individualword. This helps to cut down on ambiguous terms:“basketball coach” vs. “coach driver”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram-Based SMT

N-grams come into the picture in two ways:

1 Evaluating multiple probable translationsSimilarly to Speech Recognition, each choice is evaluatedagainst a language model

2 N-grams as “words”

As well as considering individual words, each n-gram isconsidered as a possible “phrase”, and treated as an individualword. This helps to cut down on ambiguous terms:“basketball coach” vs. “coach driver”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram-Based SMT

N-grams come into the picture in two ways:

1 Evaluating multiple probable translationsSimilarly to Speech Recognition, each choice is evaluatedagainst a language model

2 N-grams as “words”As well as considering individual words, each n-gram isconsidered as a possible “phrase”, and treated as an individualword.

This helps to cut down on ambiguous terms:“basketball coach” vs. “coach driver”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

N-Gram-Based SMT

N-grams come into the picture in two ways:

1 Evaluating multiple probable translationsSimilarly to Speech Recognition, each choice is evaluatedagainst a language model

2 N-grams as “words”As well as considering individual words, each n-gram isconsidered as a possible “phrase”, and treated as an individualword. This helps to cut down on ambiguous terms:“basketball coach” vs. “coach driver”.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Moses

Moses is an Open Source SMT system. Moses has a distinctadvantage over several other SMT systems:

1 It’s Open Source

Actively developed, and supported by a large community

2 Factored ModelsMoses is able to make use of linguistic information.

3 Open DataThe Moses developers also recognise the importance of FreeLinguistic Data, and have provided the EuroParl corpus sothat others may build a statistical MT system using it. Also,the JRC Acquis – the corpus of EU legal text (and most ofthe data behind Google Translate’s support for most officialEU languages) have prepared their corpus for use with Moses.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Moses

Moses is an Open Source SMT system. Moses has a distinctadvantage over several other SMT systems:

1 It’s Open SourceActively developed, and supported by a large community

2 Factored ModelsMoses is able to make use of linguistic information.

3 Open DataThe Moses developers also recognise the importance of FreeLinguistic Data, and have provided the EuroParl corpus sothat others may build a statistical MT system using it. Also,the JRC Acquis – the corpus of EU legal text (and most ofthe data behind Google Translate’s support for most officialEU languages) have prepared their corpus for use with Moses.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Moses

Moses is an Open Source SMT system. Moses has a distinctadvantage over several other SMT systems:

1 It’s Open SourceActively developed, and supported by a large community

2 Factored Models

Moses is able to make use of linguistic information.

3 Open DataThe Moses developers also recognise the importance of FreeLinguistic Data, and have provided the EuroParl corpus sothat others may build a statistical MT system using it. Also,the JRC Acquis – the corpus of EU legal text (and most ofthe data behind Google Translate’s support for most officialEU languages) have prepared their corpus for use with Moses.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Moses

Moses is an Open Source SMT system. Moses has a distinctadvantage over several other SMT systems:

1 It’s Open SourceActively developed, and supported by a large community

2 Factored ModelsMoses is able to make use of linguistic information.

3 Open DataThe Moses developers also recognise the importance of FreeLinguistic Data, and have provided the EuroParl corpus sothat others may build a statistical MT system using it. Also,the JRC Acquis – the corpus of EU legal text (and most ofthe data behind Google Translate’s support for most officialEU languages) have prepared their corpus for use with Moses.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Moses

Moses is an Open Source SMT system. Moses has a distinctadvantage over several other SMT systems:

1 It’s Open SourceActively developed, and supported by a large community

2 Factored ModelsMoses is able to make use of linguistic information.

3 Open Data

The Moses developers also recognise the importance of FreeLinguistic Data, and have provided the EuroParl corpus sothat others may build a statistical MT system using it. Also,the JRC Acquis – the corpus of EU legal text (and most ofthe data behind Google Translate’s support for most officialEU languages) have prepared their corpus for use with Moses.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Moses

Moses is an Open Source SMT system. Moses has a distinctadvantage over several other SMT systems:

1 It’s Open SourceActively developed, and supported by a large community

2 Factored ModelsMoses is able to make use of linguistic information.

3 Open DataThe Moses developers also recognise the importance of FreeLinguistic Data, and have provided the EuroParl corpus sothat others may build a statistical MT system using it. Also,the JRC Acquis – the corpus of EU legal text (and most ofthe data behind Google Translate’s support for most officialEU languages) have prepared their corpus for use with Moses.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy

:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipeline

Each piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”

(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)

Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replaced

The apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data

: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries

,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows

yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet.

Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering!

;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium is an Open Source Machine Translation platform.

Rule Based

Statistical disambiguation

Follows the UNIX philosophy:The system is a pipelineEach piece “does one thing, and does it well”(Not quite: analysis and generation of words are performed byseparate modes of the same program)Each component can be easily replacedThe apertium program itself is just a shell script that callsthe correct pipeline.

Several statistics-based tools for building data: dictionaries,rules

Doesn’t run on Windows yet. Shame on you for cheering! ;)

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Some Errors

(Spanish – English)

Fondo Monetario InternacionalInternational Monetary bottom

(Catalan – English)Fidel CastroFaithful Castrate

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Some Errors

(Spanish – English)Fondo Monetario Internacional

International Monetary bottom

(Catalan – English)Fidel CastroFaithful Castrate

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Some Errors

(Spanish – English)Fondo Monetario InternacionalInternational Monetary bottom

(Catalan – English)Fidel CastroFaithful Castrate

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Some Errors

(Spanish – English)Fondo Monetario InternacionalInternational Monetary bottom

(Catalan – English)

Fidel CastroFaithful Castrate

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Some Errors

(Spanish – English)Fondo Monetario InternacionalInternational Monetary bottom

(Catalan – English)Fidel Castro

Faithful Castrate

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Some Errors

(Spanish – English)Fondo Monetario InternacionalInternational Monetary bottom

(Catalan – English)Fidel CastroFaithful Castrate

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium was born from a set of translators developed inUniversitat d’Alacant, as part of the OpenTrad project.

Originallydesigned to translate between the Romance languages of Spain, ithas been expanded over time to support more distant languages:First English–Catalan More recently, Basque to Spanish

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium was born from a set of translators developed inUniversitat d’Alacant, as part of the OpenTrad project. Originallydesigned to translate between the Romance languages of Spain, ithas been expanded over time to support more distant languages:

First English–Catalan More recently, Basque to Spanish

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium was born from a set of translators developed inUniversitat d’Alacant, as part of the OpenTrad project. Originallydesigned to translate between the Romance languages of Spain, ithas been expanded over time to support more distant languages:First English–Catalan

More recently, Basque to Spanish

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Apertium

Apertium was born from a set of translators developed inUniversitat d’Alacant, as part of the OpenTrad project. Originallydesigned to translate between the Romance languages of Spain, ithas been expanded over time to support more distant languages:First English–Catalan More recently, Basque to Spanish

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Community

As part of the OpenTrad project, Apertium had a community ofdevelopers, but limited to university and business developments.Thanks mostly to Francis Tyers, Apertium has in recent yearsbegun to also acquire a community of volunteer contributors.

The first release from the volunteer community was our Welsh toEnglish translator (mostly designed by Kevin Donnelly – who alsomaintains the Welsh spell checking data).This summer, we took part in Google’s Summer of Codeprogramme, with 8 successful students. One of the translatorsdeveloped during GSoC, for Norwegian Bokmal–Nynorsk, has(within a month of release) been used to translate 30 articles onthe Nynorsk Wikipedia.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Community

As part of the OpenTrad project, Apertium had a community ofdevelopers, but limited to university and business developments.Thanks mostly to Francis Tyers, Apertium has in recent yearsbegun to also acquire a community of volunteer contributors.The first release from the volunteer community was our Welsh toEnglish translator (mostly designed by Kevin Donnelly – who alsomaintains the Welsh spell checking data).

This summer, we took part in Google’s Summer of Codeprogramme, with 8 successful students. One of the translatorsdeveloped during GSoC, for Norwegian Bokmal–Nynorsk, has(within a month of release) been used to translate 30 articles onthe Nynorsk Wikipedia.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Community

As part of the OpenTrad project, Apertium had a community ofdevelopers, but limited to university and business developments.Thanks mostly to Francis Tyers, Apertium has in recent yearsbegun to also acquire a community of volunteer contributors.The first release from the volunteer community was our Welsh toEnglish translator (mostly designed by Kevin Donnelly – who alsomaintains the Welsh spell checking data).This summer, we took part in Google’s Summer of Codeprogramme, with 8 successful students. One of the translatorsdeveloped during GSoC, for Norwegian Bokmal–Nynorsk, has(within a month of release) been used to translate 30 articles onthe Nynorsk Wikipedia.

Jimmy O’Regan Machine Translation and Translation Technology

OutlineTools for TranslatorsFree Language DataMachine Translation

Is Machine Translation a Translator’s Tool?N-GramsMosesApertium

Language Pairs Supported

Spanish – Catalan Spanish – RomanianFrench – Catalan Occitan – CatalanEnglish – Galician Occitan – SpanishSpanish – Portuguese English – CatalanEnglish – Spanish English – EsperantoSpanish – Galician French – SpanishEsperanto – Spanish Welsh – EnglishBreton – French Esperanto – CatalanPortuguese – Catalan Portuguese – GalicianBasque – Spanish Nynorsk – Bokmal

Jimmy O’Regan Machine Translation and Translation Technology

Recommended