D3.2.2: B2B INTEGRATION SHOWCASE · The advantages of using an external ... localization workflow of the contents benefits from each ... ‐ DRUPAL is a free and

D3.2.2: B2B INTEGRATION SHOWCASE

Mauricio del Olmo, Laura Guerrero, Pedro L. Díez Orzas, Giuseppe

Deriard, Pablo Badía (Linguaserve)

Karl Fritsche, Clemens Weins, Stephan Walter (Cocomore)

Distribution: Public

MultilingualWeb‐LT (LT‐Web) Language Technology in the Web

FP7‐ICT‐2011‐7

Project no: 287815

2D3.2.2

Document Information

Deliverable number: 3.2.2

Deliverable title: B2B Integration Showcase

Dissemination level: RE

Contractual date of delivery: 30th September 2013

Actual date of delivery: 28th October 2013

Author(s): Mauricio del Olmo, Laura Guerrero, Pedro L. Díez Orzas, Giuseppe Deriard, Pablo Badía, Karl Fritsche, Clemens Weins, Stephan Walter, XXX, XXX

Participants: Cocomore, Linguaserve

Internal Reviewer: Linguaserve

Workpackage: WP3

Task Responsible: Giuseppe Deriard

Workpackage Leader: Pedro L. Díez Orzas

Revision History

Revision Date Author Organization Description

1 05/2013 Mauricio del Olmo Linguaserve Draft creation and initial writing and compiling of information

2 06/2013 Stephan Walter Cocomore Section 5 and added Cocomore specific content

3 06/2013 Pedro L. Díez Orzas Linguaserve Draft restructuring and completion

4 06/2013 Mauricio del Olmo Linguaserve Added examples in section 5 and more completion

5 07/2013 Stephan Walter Cocomore Sections 4 and 6

6 09/2013 Pedro L. Díez Orzas Linguaserve Final revision and approval

7 10/2013 Pedro L. Díez Orzas Linguaserve Minor final modifications and final approval

3D3.2.2

CONTENTS

Document Information ....................................................................................................................................... 2

Revision History .................................................................................................................................................. 2

Contents ............................................................................................................................................................. 3

1. Executive Summary .................................................................................................................................... 4

2. Introduction ................................................................................................................................................ 5

3. CMS and TMS integration with ITS 2.0 ....................................................................................................... 6

4. ITS 2.0 implementation in the CMS ............................................................................................................ 9

4.1. ITS 2.0 aware translation workflow in the CMS .................................................................................... 9

4.1.1. Workflow management with TMGMT .............................................................................................. 9

4.1.2. ITS 2.0 Annotation .......................................................................................................................... 10

4.1.2.1. Local Markup via WYSIWYG ....................................................................................................... 10

4.1.2.2. Support for Global Markup ........................................................................................................ 11

4.1.2.3. Annotation as a separate workflow step.................................................................................... 11

4.1.2.3.1. Functionality .......................................................................................................................... 11

4.1.2.3.2. User Interface......................................................................................................................... 11

4.1.2.3.3. Implementation ..................................................................................................................... 12

4.1.2.3.4. Data categories with automatically determined values ........................................................ 12

5. ITS 2.0 implementation in the TMS .......................................................................................................... 13

5.1. Pre‐production phase .......................................................................................................................... 14

5.2. Translation and revision phase ........................................................................................................... 15

5.3. Post‐production phase ........................................................................................................................ 16

5.4. Format transformations in the l10n workflow .................................................................................... 18

6. Glossary of terms and acronyms .............................................................................................................. 20

7. References ................................................................................................................................................ 21

8. APPENDIX I: TMS ITS 2.0 Implementation ................................................................................................ 22

9. APPEDIX II: ACCESS AND USER GUIDE ...................................................................................................... 29

9.1. CMS Side (Cocomore AG) .................................................................................................................... 29

9.2. TMS side (Linguaserve) ....................................................................................................................... 34

4D3.2.2

1. EXECUTIVE SUMMARY

The present document describes the B2B Integration Showcase whose objective is to demonstrate the

applicability of LT‐Web ITS 2.0 metadata in a CMS to TMS localization chain.

The sections 3 to 5 describe how integration works between the two sides and several aspects related with the

implementation of ITS 2.0 data categories.

After that, the appendix I ‘TMS ITS 2.0 Implementation’ describes how each data category is implemented and

the appendix II ‘Access and user guide’ describes briefly how to access and how they work.

5D3.2.2

2. INTRODUCTION

The large volume of information and web content justifies the use of CMS systems for medium to big size

companies and organizations. CMS systems provide benefits such as content control, several user profiles,

abstraction and workflows.

When we insert the multilingual variable to the CMS picture, a translation workflow is highly recommended.

The advantages of using an external localization provider and CAT tools gives added value such as the use of

translation memories, glossaries and the experience with translation management.

The showcase will exemplify how ITS 2.0 allows more integration between the two sides and how the

localization workflow of the contents benefits from each data category implemented.

Additional information and figures on the data used in our test case and on the annotations performed in the

source and multilingual content can be found in D3.2.3, section 4.1.1.2.

6D3.2.2

3. CMS AND TMS INTEGRATION WITH ITS 2.0

For this showcase, Cocomore and Linguaserve have used a real customer. “VDMA ‐ Verband Deutscher

Maschinen‐ und Anlagenbau ‐ German Engineering Federation”1, and the following existing technology has

been used for the showcase in the CMS side (Cocomore):

‐ DRUPAL is a free and open‐source content management framework (CMF) written in PHP and

distributed under the GNU General Public License. Drupal version 7 was used for this

implementation.

In the TMS side (Linguaserve):

‐ Global Business Connector Server (GBC Server) is a Globalization Management System that has

been on the market since 2003. The interoperability applies Webservices and XML, and it can be

connected to any CMS, DMS, or ERP, or proprietary systems. The content and document

translation workflow is completely automated. It was developed and is owned by Linguaserve.

‐ Platform for Localization, Interoperability and Normalization of Translation (PLINT) is a

workflow control platform for translation and localization processes. It integrates the pre‐

production/post‐production engine to process formats to and from CAT tools and Machine

Translation engines.

‐ Computer Assisted Translation tool Transit XV (STAR Group).

Interoperability is achieved by the offline translation system interface between the "Client Application"

(Drupal) and Linguaserve’s translation server (Global Business Connector Server). GBC Server provides a

number of services for the translation and localization of content, as well as for the exchange of other

information necessary for the production and monitoring of orders. It provides a service with which the Client

Application can check the correct functioning of the connections (Proxy, client's Firewall, etc.) without the CA

having to simulate sending files. GBC Server uses Simple Object Access Protocol (SOAP).

The most relevant webservice operations

are:

Send file for translation

Reception confirmation

Cancellation

Check availability of translations

Download translations

Download confirmation

1 See the VDMA Web site at http://www.vdma.org/.

7D3.2.2

The content originates in a CMS, and gets exposed/serialized as XHTML + ITS 2.0. This is sent to a TMS and

processed in a workflow. Upon completion, the TMS exposes/serializes localized/translated XHTML + ITS 2.0 to

the CMS. The basic steps of an ITS 2.0‐aware content creation and translation‐ process are as follows:

VDMA has content produced in the Drupal CMS.

Before being sent, the content is annotated with ITS 2.0 metadata with means of automatic and

manual annotation. This localization workflow is an XML based tool chain; hence, XHTML + ITS 2.0 is

used as the interchange format.

The content is sent to the Linguaserve Global Business Connector Contents (GBCC) translation server,

processed in Linguaserve’s internal localization workflow “Platform for Localization, Interoperability

and Normalization of Translation” (PLINT).

The ITS 2.0 metadata is used during the LSP (Linguaserve) internal processing for several

localization tasks (providing context to the translators, blocking the non‐translatable

contents in the CAT tool, selecting terminology and translation memories…) and also

updated in some cases as a result of the process (Provenance: the translator and reviser that

have carried out the job).

After, once the annotated content is translated and the metadata is treated, they are downloaded by

the client and imported into the CMS.

This integrated approach affects almost all areas of the traditional translation workflow. Accordingly, it

requires modifications and extensions throughout the tool chain. The following graphic shows a vision of the

architectural entities that are involved in ITS 2.0‐aware content and translation handling:

Features of the solution that we created based on this architecture can be assigned to either the content

provider’s or the LSP side of the picture. On the content provider’s side the creation of the ITS 2.0 metadata

aware workflow involves the following areas:

Annotation of source language content with ITS 2.0 metadata within the Drupal CMS. Structural

annotation rules can be specified as global rules on page/content type level, while local metadata is

added by hand. In addition, automated annotation tools can be integrated through a standardized

interface to support the user in creating such local markup.

Manual annotation features are available in all generally expected interaction modes (toolbar

buttons, context menu, keyboard shortcuts):

8D3.2.2

Two annotation approaches are supported:

a) Annotation may be carried out as part of the content creation process, via features that have

been added as plugins to the out‐of‐the‐box Drupal WYSIWYG editor.

b) Annotation may be carried out as a separate step, without the ability to modify the content.

This allows workflows that separate content know‐how and translation management.

Transparent data round‐tripping: Triggered from within Drupal, this is carried out in the background

via export/import of XHMTL+ITS 2.0 markup files, to be sent to/received from LSP. The process is

based on an extended version of the Drupal Translation Management (TMGMT)‐module.

Translation review: ITS 2.0 markup is retained in this step so that annotated information can be taken

into account for QA purposes.

On the LSP side, the creation of the ITS 2.0 metadata aware workflow encompasses three areas:

Pre‐production/post‐production engine for processing content files annotated with ITS 2.0.

LSP internal localization workflow to provide support to project management and production

processes.

Computer Assisted Translation (CAT) tool usage for translation, revision and postediting with ITS 2.0

annotated content.

The following diagram illustrates the life cycle of each data category in the complete roundtrip:

9D3.2.2

4. ITS 2.0 IMPLEMENTATION IN THE CMS

Cocomore integrated ITS 2.0 into the open‐source Content Management System (CMS) Drupal. This required

the development and adaptation of several modules (see D3.1.1 for more information):

‐ Drupal TMGMT‐module (extension to allow workflows with ITS 2.0 annotation)

‐ Drupal WYSIWYG editor: Plugin for ITS 2.0 annotation

‐ JQuery plugin for ITS 2.0 annotation in a separate step (new implementation)

‐ Interfacing with Global Business Connector Contents (GBCC) and web services (implementation of

data export/import and client implementation)

4.1. ITS 2.0 aware translation workflow in the CMS

This section describes the translation workflow and the ITS 2.0 handling within the open‐source CMS Drupal.

Drupal can be extended easily by modules, and there are already a couple of modules available to help the

user with translation processes.

4.1.1. WORKFLOW MANAGEMENT WITH TMGMT

We used and extended the "Translation Management Tool" (TMGMT). This module provides the basic

translation workflow, which comprises the following steps:

1. Create translation job

2. Send job to translation service

3. Reintegrate translated job into Drupal

Furthermore, TMGMT is designed to work with any content and any translation service. It provides interfaces

for handling different sources and services. These are complemented by a default implementation of the

source interface, which creates TMGMT jobs from content pages (called nodes in Drupal). For strings that are

not part of Drupal nodes (such as menu links, error messages, etc.) we created an additional implementation,

which generates a TMGMT job for those untranslated strings in the Drupal CMS. To operate with the

Linguaserve Web Service we implemented a translation service for TMGMT to send TMGMT jobs to

Linguaserve and retrieve these jobs when they are finished. With these modules the general translation

workflow is covered. The described functionality is implemented in the module Drupal TMGMT Workflow (see

D3.1.1 for more information).

The TMGMT Linguaserve module which is a translation service for TMGMT handles all SOAP calls to

Linguaserve and creates an XHTML file from a TMGMT job. This XHTML file is used as exchange format

between Cocomore and Linguaserve. This file uses script‐tags for global data categories and the normal HTML

markup as described in the ITS 2.0 and only contains the content and no menu or styling information. In this

way it can be interpreted easily by other services. The described functionality is implemented in the module

Drupal TMGMT Translator Linguaserve (see ¡Error! No se encuentra el origen de la referencia. for details on

availability).

10D3.2.2

4.1.2. ITS 2.0 ANNOTATION

This is an example of a Drupal XHTML exported file with ITS 2.0 before translation:

In section 5.3, we will show the post processed file with ITS 2.0 with the same Drupal XHTML example, ready

to be sent and imported into the CMS.

4.1.2.1. LOCAL MARKUP VIA WYSIWYG

For the ITS 2.0 integration we had to develop another module. This module provides the integration of ITS data

categories into Drupal. It extends the WYSIWYG editor with new buttons to allow the user to add and edit local

ITS markup in content pages. The following ITS data categories can be set with the WYSIWYG while creating or

editing a content page:

<?xml version="1.0" encoding="UTF-8"?> <html xmlns="http://www.w3.org/1999/xhtml" xmlns:its="http://www.w3.org/2005/11/its" xmlns:itsx="http://www.w3.org/2008/12/its-extensions" its:version="2.0"> <head> <meta name="DC.subject" content="'Angewandte Wissenschaft', 'Unternehmen', 'Maschinenbau', 'Allgemein', 'Anlagenbau', 'Kunststoff- und Gummimaschinen', 'Technologien'"/> <script type="application/its+xml"> <its:rules> <its:domainRule selector="/h:html/h:body" domainPointer="/h:html/h:head/h:meta[@name='DC.subject']/@content"/> <its:locNoteRule locNoteType="description" selector="/h:html/h:body"> <its:locNote translate="no">Pressemitteilung</its:locNote> </its:locNoteRule> <itsx:readinessRule ready-at="21/01/2013 13:48:56:000 CET" priority="1/3" complete-by="19/02/2013 16:00:00:000 CET" ready-to-process="hTranslate, reviseQA, hReview, publish"/> </its:rules> </script> </head> <body id="36672" lang="de"> <div id="36672-node_title" its-allowed-characters="[^<>]" its-storage-size="255">VDMA verstärkt den Kampf gegen Produktpiraterie</div> <div id="36672-body-0-value" its-allowed-characters="."> <p>17.05.2010 - Der Kampf gegen Produktpiraterie muss an vielen Fronten gleichzeitig geführt werden. Eine wichtige Maßnahme imgegen die Verletzung von materiellem und geistigem Eigentum ist die juristische Unterstützung. Der <span translate="no">VDMA</span> setzt sich mit aller Vehemenz für den SchutzMitgliedsunternehmen ein.</p> <p>„<span its-loc-note="Bitte korrekte sinngemäße Übersetzung mit Marc Wiesner absprechen." its-loc-note-type="description">Das Internet macht vieles- beispielsweise auch die Verletzung von Schutzrechten</span>", betont <span translate="no">Marc Wiesner</span>, Experte für Produktpiraterie der Abteilung Recht im <span translate="no">VDMA</span>. Viel schneller als früher bemerken es die Unternehmensvertreter heutzutage, wenn Produkte angeboten werden, die den eigenen täuschend ähnlich sind oderNachahmungen darstellen. Das weltweite Datennetz hilft nicht nur beim Verkauf illegaler Waren, es bringt ebenso Rechtsverletzungen schnell und überall zutage.</p> <p>Der <span translate="no">VDMA</span> bietet seinen Mitgliedsunternehmen zusätzlich zu Publikationen und Informationen bei Veranstaltungen rechtliche Beratung speziell zu Verletzung<span lang="en">Know-how</span> und gewerblichen Schutzrechten an.</p> </div> <div id="36672-body-0-format" translate="no" its-allowed-characters=".">full_html</div> </body> </html>

11D3.2.2

Translate

Locale Filter

Text Analysis

Localization Note

Language Information

Directionality

Terminology

The described functionality is implemented in the module Drupal ITS2.0 Integration (see D3.1.1 for details on

availability).

4.1.2.2. SUPPORT FOR GLOBAL MARKUP

Apart from the possibility of setting these data categories as local markup, there are also a few data categories

that can act as global markup. Support for such global markup is managed on a per‐content‐type basis.

Enabling ITS support for a given content type creates a new field set in the edit form for this type of content.

The field set can be used to enter global XPath rules. It is possible to set default global rules for each content

type or globally for the complete site.

The following data categories are available for global markup:

Domain

Translate

Localization Note

Revision/Translation Agent (from the Provenance data category)

The described functionality is also implemented in the module Drupal ITS2.0 Integration, but has to be enabled

manually after installation (see D3.1.1 for more information).

4.1.2.3. ANNOTATION AS A SEPARATE WORKFLOW STEP

4.1.2.3.1. FUNCTIONALITY

As an extension to the normal WYSIWYG editor in the content edit form, we added a new "Language

Management" form. The form provides an editor in which it is only possible to work on (add, remove change)

the ITS 2.0 markup of a node, while the actual content is all write‐protected. This supports a separation of

content editing and ITS 2.0 annotation into two distinct workflow steps: A special user role (e.g. a translation

manager) can add ITS data very easily after content creation without accidentally changing the content itself.

This role will also be able to see and can edit the global markup.

4.1.2.3.2. USER INTERFACE

Local and global markup can be highlighted separately in the content. This is controlled in the UI by means of

checkboxes. In this way, the user can choose what he/she wants to see and doesn't get overwhelmed with all

12D3.2.2

data categories at once. If the user selects content next to the selection, a small window pops up. In this

window the user can choose a data category to add to the selected content. There are also keyboard shortcuts

available for the data categories to support even faster tagging. For the simple data categories like "Translate"

they just add the attribute with the contextually most likely value. For instance, a translate attribute will be set

to the negated value of the one valid in the context, thus translate=”no” within text that is not in the scope of

any other translate attribute (because the default translate=”yes” is assumed for such text). For all other data

categories, a new modal window appears where the needed data can be edited, like the note and note type

for "Localization Note".

4.1.2.3.3. IMPLEMENTATION

The functionality described above depends highly on JavaScript and is built on top of the ITS2.0 jQuery plugin

(see D3.1.1 for details on availability), which was developed by Cocomore as well. This jQuery plugin provides

functionality for the selection of text nodes with special data category values, and getting the ITS values of a

text node. It is released independently of the Drupal modules, so other frameworks or users can use it in their

implementations as well. For example, a programmer can quickly get all non‐translatable text nodes from an

HTML and XHTML page to add special styles to it. The plugin correctly handles both local and global markup,

including global markup in a script tag and external linked global markup. The modules have been tested

successfully against the ITS 2.0 test suite.

4.1.2.3.4. DATA CATEGORIES WITH AUTOMATICALLY DETERMINED VALUES

There are several data categories that have a special status when integrating ITS 2.0 in a CMS by the fact that

they allow for an especially high degree of integration. This may be because the CMS provides specific means

for handling out of the box, or because adequate values can be derived automatically from other information

that is available from various sources within the CMS and workflow. This special status is also reflected in the

Drupal ITS2.0 integration module.

For the Domain data category you can select that it shouldn't be a text field where the user can type in the

domain, instead you can use the taxonomy system from Drupal. With this you can create your own vocabulary

or use an existing one and just select the domains on content editing. The Provenance data can't be edited by

the user, it just shows and stores this information and will be automatically set by the translation service. In a

similar manner, additional data categories are embedded in the translation process. Data categories like

Allowed Characters, Storage Size and Readiness from the ITS extension will be added automatically to the

content sent to the LSP depending on Drupal’s field definitions of a particular field. As an example there is a

maximum length of 255 characters for the title field, and in this case the storage size category is added to the

title field with the respective values set. The user doesn’t have to care about this at all. As other examples, the

expected finalization date and priority are added by the translation manager before the translation job is

submitted to the LSP.

The described functionality is implemented in the module Drupal ITS2.0 Integration (see D3.1.1 for details on

availability).

13D3.2.2

5. ITS 2.0 IMPLEMENTATION IN THE TMS

This section explains which ITS 2.0 data category has been implemented, its usage and its application on the

different phases of the localization workflow. There are also explanatory details about the implementation and

examples of ITS metadata.

In the LSP side, the creation of the ITS 2.0 metadata aware workflow encompasses three areas:

Pre‐production/post‐production engine for processing content files annotated with ITS 2.0.

LSP internal localization workflow to provide support to project management and production

processes.

Computer Assisted Translation (CAT) tool usage for translation, revision and postediting with ITS 2.0

annotated content.

The ITS 2.0 Data categories implemented in the TMS side are:

Translate Specifies whether the content should be translated or not.

Localization note Used to communicate notes to localizers about a particular item ofthe content.

Domain Used to identify the domain of the content.

Language information Used to express the language of a given piece of content.

Allowed characters Used to specify what characters are allowed in a given content.

Storage size Used to specify the maximum storage size of a given content.

Provenance Provides information concerning how a text was translated (e.g., MT, human translation) and revised (e.g., human post‐editing).

Readiness (*) Used to indicate the readiness, the relative priority and an expectation of when a specific process should be completed.

(*) "Readiness" is not part of ITS 2.0 since, in the given time frame, the W3C MultilingualWeb‐LT Working

Group could not find consensus on all aspects of Readiness. The implementation of Readiness as an extension

to ITS 2.0 allows to gather experience and to consider this data category for a future version of ITS. This

approach already helped after the creation of ITS 1.0 to develop other data categories that are now a stable

part of ITS 2.0.

14D3.2.2

5.1. Pre-production phase

The original file is processed with the ITS 2.0 engine that reads the data categories within and the results are:

ITS 2 localization workflow interaction:

Localization Note: when alert type, send a notification to the project manager and add tooltip visualization in the workflow.

Language Information: quality check to ensure the source language content is according to the Webservice parameter.

Storage Size: quality check for the original content.

Readiness: control of processes to be carried out. Date control for availability and delivery is registered. Priority control.

The data category treatment in the internal pre‐production phase is:

Pre‐production phase

Data category L10N workflow XHTML Global XHTML Local

Translate Omit selected not translatable contents.

A particular node could be non‐translatable. Mark parts of the content marked as non‐translatable for blocking.

Localization Note

When alert type, send a notification to the project manager and add tooltip visualization in the workflow.

Create reference node to inform the translator.

Inform the translator.

Domain Automatic selection of terminology and translation memories.

Create reference node to inform the translator.

Language information

Quality check to ensure the source language content is according to the webservice parameter.

Inform the translator.

Allowed Characters

15D3.2.2

Pre‐production phase


Storage size Quality check for the original content. Inform the translator.

Provenance

Possibility to reassign the same translator/reviewer in new versions of the same content (based on identifiers).

Readiness (*) Priority checked with webservice.Set expected finalization date.

(*) ITS 2.0 extension.

5.2. Translation and revision phase

The CAT tool oriented XML file resulting from the previous step is used here. The ITS 2.0 localization workflow interaction can be described as:

Domain: automatic selection of CAT terminology and dictionaries. Selection of Translation Memories by domains.

Provenance: possibility to reassign the same translator/reviewer in new versions of the same content (based on identifiers). Inform the project manager.

This file will be imported in the CAT tool using a filter created ad hoc, based on an XML file with embedded

HTML tags filter and modified to add new special tags. The content inside the xlasRefTrad nodes is reference

content for the translators/reviewers. It is visible but blocked in the CAT tool. The content inside the xlasTrad

nodes is translatable content. It is editable by the translators/reviewers except the standard HTML tags and

the content between the special inline blocking tags (<xlasbloq>).

16D3.2.2

The specific manipulation with each data category is the following:

Translate: The content of the XHTML div nodes with the local attribute translate="no" is not extracted. Additionally, in the translatable div nodes with HTML content the preprocessing step adds the xlasbloq tags before and after the pieces of the content that are not translatable.

Localization Note: The content of the localization note (from the <its:locNote> node) is added in an xlasRefTrad node, blocked by the CAT filter but visible for the translators and reviewers, if applicable (selector attribute). Additionally, the contents of the its‐loc‐note attributes (local usage) are blocked by the CAT tool.

Domain: The content of the domain attribute (as established by the domain pointer "/h:html/h:head/h:meta[@name='DC.subject']/@content") is added in an xlasRefTrad node, blocked by the CAT filter but visible for the translators and reviewers, if applicable (selector attribute).

Language Information: The local lang attributes are visible for the translators and reviewers, but also blocked. Workflow usage: The source language information is obtained from the DB (originally a web service parameter), always available for the LSP‐based Translation Process Managers and used to select the translators and reviewers.

Storage Size: The local its‐storage‐size attribute value is obtained and informed in the xMax attribute of the xlasTrad node when applicable. Also the size of the original content is calculated and informed in the xMaxOrig attribute.

Readiness: Workflow usage: the priority info is obtained from the DB (originally a web service parameter) and always available for the LSP‐based Translation Process Managers. The expected finalization date (complete‐by parameter) is updated into the system DB in the preprocessing step.

How the translators and proofreaders work in the CAT tool with all this information is described in the section

Translator’s Training.

5.3. Post-production phase

The XML translated file is exported form the CAT tool and advances to the next step in the localization

workflow.

17D3.2.2

ITS 2.0 localization workflow interaction:

Domain: Storage and classification of Translation Memories by domains.

Readiness: control of processes to be carried out. Date control for availability and delivery.

The data category treatment in the internal post‐production phase is:

Post‐production phase


Translate Undo blocking of parts of the content marked as non‐translatable.

Insert translation on the translatable nodes. Undo blocking of parts of the content marked as not translatable.

Localization Note

Domain Storage and classification of translation memories by domains.

Language Information

Update the lang attribute in the translated nodes. Update the language attributes in the translated content.

Allowed Characters

Restriction compliance check.

Storage size Limitation compliance check.

Provenance Add or update the data category attributes.

Readiness (*) Date control for availability and delivery.

Update the data category node.

(*) ITS 2.0 extension.

This file is exported from the CAT tool and post‐processed in the internal localization workflow. The specific manipulation with each data category is the following:

Translate: Insert the translation on the XHTML translatable nodes. Additionally, undo the blocking of parts of the content marked as non‐translatable (remove the xlasbloq tags).

Language Information: Update the lang attribute in the body node. Change the code of the source language with the code of the target language. Also update the local language attributes in the translated contents.

Allowed Characters: The post‐processing engine checks if the content of the node fulfills the restriction indicated by the its‐allowed‐characters attribute.

Storage Size: The post‐processing engine checks if the content of the node fulfills the restriction indicated in the its‐storage‐size attribute.

Provenance: The global data category tag is added or updated showing the internal ID of the translator (its‐person) and reviewer (its‐rev‐person) who has carried out the job and with the name of the LSP company (its‐org and its‐rev‐org).

Readiness: The ready‐to‐process attribute is updated deleting the processing steps already carried out and leaving only the next steps in the localization chain (hReview, publish). These tasks will be executed in the client CMS side: the import and publication of the translated content. Also the ready‐at attribute is updated with the time stamp.

At this point the translated file is available to be downloaded from the client’s CMS side.

18D3.2.2

Example of a translated and post‐processed Drupal XHTML file:

5.4. Format transformations in the l10n workflow

Several approaches where made before the final decision was taken about the interchange format:

1.‐ XML with CDATA: The first approach was based on the use of an XML generated from Drupal. The necessity

to store HTML content inside and to maintain a direct relation between the content of the nodes and the fields

in the CMS side, contributed to the decision of the use of CDATA sections. But that originated a problem with

the global rules because the xpaths of the global rules could not be used to select content inside the CDATA

sections. Another aspect that was desirable to be avoided was not using an external rules file for the global

rules, because that created dependencies between several files, like having to send two files from the CMS

side to the LSP side for each content to be translated. For example, in case of communication problems one of

them could be lost.

<?xml version="1.0" encoding="UTF-8"?> <html its:version="2.0" xmlns="http://www.w3.org/1999/xhtml" xmlns:its="http://www.w3.org/2005/11/its" xmlns:itsx="http://www.w3.org/2008/12/its-extensions"> <head> <meta content="'Allgemein', 'Angewandte Wissenschaft', 'Unternehmen', 'Maschinenbau', 'Anlagenbau', 'Kunststoff- und Gummimaschinen', 'Technologien'" name="DC.subject"/> <script type="application/its+xml"> <its:rules> <its:domainRule domainPointer="/h:html/h:head/h:meta[@name='DC.subject']/@content" selector="/h:html/h:body"/> <its:locNoteRule locNoteType="description" selector="/h:html/h:body"> <its:locNote translate="no">Pressemitteilung</its:locNote> </its:locNoteRule> <itsx:readinessRule complete-by="19/02/2013 16:00:00:000 CET" priority="1/3" ready-at="30/01/2013 17:46:27:744 CET" ready-to-process="hReview, publish"/> </its:rules> </script> </head> <body id="36672" its-org="Linguaserve" its-person="21686" its-rev-org="Linguaserve" its-rev-person="20697" lang="fr"> <div id="36672-node_title" its-allowed-characters="[^<>]" its-storage-size="255">VDMA renforce la lutte contre le piratage des produits</div> <div id="36672-body-0-value" its-allowed-characters="."> <p>17/05/2010 - La lutte contre le piratage des produits doit être menée sur de nombreux fronts à la fois. L'assistance juridique est une mesure dans la lutte contre la violation de la propriété matérielle et intellectuelle. <span translate="no">VDMA</span> milite avec véhémence la protection de ses sociétés membres.</p> <p>« <span its-loc-note="Bitte korrekte sinngemäße Übersetzung mit Marc Wiesner absprechen." its-loc-note-type="description">Internet donne de transparence à beaucoup de choses, dont notamment à la violation du droit d'auteur</span> », souligne <span translate="no">Marc Wiesner</span>, écialiste en piratage des produits au sein du département juridique de <span translate="no">VDMA</span>. De nos jours, les représentants des entreprises remarquent beaucoup plus rapidement qu'autrefois la frappante entre les produits proposés et les leurs ou les contrefaçons illégales. Même si le réseau mondial de données aide à la vente de illégales, il met également au grand jour, rapidement et en tout lieu, des violations du droit.</p> <p><span translate="no">VDMA</span> propose également à ses sociétés membres un conseil juridique en matière de publications et 'informations diffusées lors de manifestations, particulièrement en matière de violation du <span lang="fr">savoir-faire</span> des droits de propriété industrielle.</p> </div> <div id="36672-body-0-format" its-allowed-characters="." translate="no">full_html</div> </body> </html>

19D3.2.2

2.‐ HTML 5: It was briefly considered to use HTML 5 as interchange format, but it was discarded due to the

better XHTML capabilities as interchange format.

3.‐ XHTML with global rules: In a third approach, the change was to use XHTML format. That format provided

the advantages that it could be used as an XML for the interchange of contents between the two sides (CMS ‐

LSP), that it allowed the use of global rules inside the file and that it enabled to put the HTML contents inside

the file directly as part of the tree of the document without CDATA.

4.‐ XHTML with extension and script tag: Some changes were made because of the evolution in the ITS 2.0

standard to stabilize the extension mechanism (needed for the Readiness metadata). Also a change was made

to follow the best practices related to the use of global rules inside the content (within a script tag). The global

rules inside the script tag use the ITS xml syntax and the local rules use the HTML syntax. This decision was

taken in order to make fewer transformations, as the final content is outputted from Drupal and into web

browsers as HTML 5.

Diagram showing the details of the transformations of the contents inside the l10n workflow:

The appendix I 'TMS ITS 2.0 Implementation' describes how each data category is implemented.

Original file XHTML

+ ITS 2.0

Pre‐processed

file CAT tool

oriented XML

Translated file CAT tool

oriented XML

Final file XHTML +

ITS 2.0 translated

file

20D3.2.2

6. GLOSSARY OF TERMS AND ACRONYMS

Term/Acronym Definition

B2B Business‐to‐business: electronic communications between businesses or enterprises.

CAT Tool Computer Aided Translation tool.

CMS Content Management System.

Drupal An open‐source Content Management System (CMS).

GBCC Global Business Connector Contents (Linguaserve’s software)

ITS The Internationalization Tag Set (ITS) is a set of attributes and elements designed to provide internationalization and localization support in HTML5 and XML documents.

JQuery A multi‐browser JavaScript library designed to simplify the client‐side scripting of HTML.

HTML HyperText Markup Language.

L10N Localization

LSP Language Service Provider

PLINT Platform for Localization, Interoperability and Normalization of Translation (Linguaserve’s software)

TMS Translation Management System.

VDMA Verband Deutscher Maschinen‐ und Anlagenbau (German Engineering Federation)

WYSIWYG What You See Is What You Get

XHTML eXtensible HyperText Markup Language.

XML eXtensible Markup Language

XPath XPath is a language for addressing parts of an XML document, designed to be used by both XSLT and XPointer.

21D3.2.2

7. REFERENCES

Apache OFBiz:

http://ofbiz.apache.org/

Cocomore

www.cocomore.com

Drupal Official Web:

http://drupal.org/

ITS Interest Group:

http://www.w3.org/International/its/ig/

ITS 2.0 Requirements:

http://www.w3.org/TR/its2req/

ITS 2.0 Tag Set:

http://www.w3.org/TR/its20/

Linguaserve

www.linguaserve.com

MultilingualWeb official web:

http://www.multilingualweb.eu/

MultilingualWeb‐LT EC Working Group Project Wiki:

http://www.w3.org/International/multilingualweb/lt/

SOAP specifications:

http://www.w3.org/TR/soap/

STAR Transit:

http://www.star‐group.net/ENU/group‐transit‐nxt/transit.html

VDMA

www.vdma.org

VDMA Machines for Plastics:

http://www.machines‐for‐plastics.com/kug/

W3C XHTML 1.0 recommendation:

http://www.w3.org/TR/xhtml1/

W3C XML 1.0 recommendation

http://www.w3.org/TR/xml/

W3C XPath 1.0 recommendation

http://www.w3.org/TR/xpath/

22D3.2.2

8. APPENDIX I: TMS ITS 2.0 IMPLEMENTATION

TRANSLATE

TRANSLATE IN THE PRE‐PRODUCTION PHASE

A method obtains the ITS global rules (Its2Implementation.getItsGlobalRulesElement()). Another method

obtains the global translatable rules from the ITS global rules (Its2Implementation.getGlobalTranslateRules()).

After that, the global translate rules (translate="yes") and the global non‐translate rules (translate="no") are

stored in two different objects.

The document nodes are traversed. For each node:

If a global translate rule applies to the node (xpath), then the current translate status is updated for direct

application and inheritance (method Its2Implementation.getGlobalTranslate()). The local translate rules, the

current translate status and the defaults are checked to know the treatment of the node

(Its2Implementation.isTranslatableElement()). Also the current status is updated accordingly for inheritance.

If the node is not translatable, jump to the next node.

If the node is translatable, mark the node as translatable. Extract the content.

Traverse the HTML content, if any, for local rules application. Add tags for blocking content with

translate="no" in the CAT tool (Its2Implementation.applyTranslateContentRules()). Put the content in the CAT

tool oriented XML in a translate node (xlasTrad).

Example:

TRANSLATE IN THE POST‐PRODUCTION PHASE

Traverse the nodes of the document. If the node was marked as translatable, recover the translation from the

translated CAT tool oriented XML (xlasTrad node).

If there is HTML mark‐up in the content, remove the marks for blocking non‐translatable parts

(Its2Implementation.undoTranslateContentRules()), insert it the translation in the document.

Example:

Der <span translate="no">VDMA</span> setzt sich mit aller Vehemenz für den SchutzMitgliedsunternehmen ein.

<span translate="no">VDMA</span> milite avec véhémence la protection de ses sociétés membres.

23D3.2.2

LOCALIZATION NOTE

LOCALIZATION NOTE IN THE PRE‐PRODUCTION PHASE

A method obtains the ITS global rules (Its2Implementation.getItsGlobalRulesElement()).

Another method obtains the global localization note rules from the ITS global rules (Its2Implementation.

getGlobalLocNoteRules ()).

A third method obtains all the alert type localization notes (Its2Implementation.getGlobalLocNoteValues()).

If there is at least one localization note of type = alert, an e‐mail is sent to the project managers and the

comments of the file are updated in the DB of the system for tooltip visualization in the localization workflow

(method Its2Implementation.applyWfLocNotesIntegration()).


If a global localization note rule applies to the node (method Its2Implementation.getAppliedLocNoteValue())

then a reference node (xlasRefTrad) is created in the CAT tool oriented XML for the translators/proof‐readers.

Examples (global and local usage):

DOMAIN

DOMAIN IN THE PRE‐PRODUCTION PHASE

A method obtains the ITS global rules (Its2Implementation.getItsGlobalRulesElement()). Another method

obtains the global domain rules from the ITS global rules (Its2Implementation. getGlobalDomainRules()). A

third one stores in the BD of the system the domains associated with the file

(Its2Implementation.applyWfPreDomainIntegration()).


If a global domain rule applies to the node (method Its2Implementation.getAppliedDomainValue()) then a

reference node (xlasRefTrad) is created in the CAT tool oriented XML for the translators/proof‐readers.

Example:

<its:locNoteRule locNoteType="description" selector="/h:html/h:body"> <its:locNote translate="no">Pressemitteilung</its:locNote> </its:locNoteRule>

<span its-loc-note="Bitte korrekte sinngemäße Übersetzung mit Marc Wiesner absprechen." its-loc-note-type="description">Internet donne de transparence à beaucoup de choses, dont notamment à la violation du droit d'auteur</span>

<meta name="DC.subject" content="'Angewandte Wissenschaft', 'Unternehmen', 'Maschinenbau', 'Allgemein', 'Anlagenbau', 'Kunststoff- und Gummimaschinen', 'Technologien'"/> […] <its:rules> <its:domainRule domainPointer="/h:html/h:head/h:meta[@name='DC.subject']/@content" selector="/h:html/h:body"/> </its:rules>

24D3.2.2

DOMAIN IN THE CAT TOOL PROJECT CREATION STEP

The domains associated with each selected file are retrieved and stored into a list. The dictionaries

corresponding with each domain are obtained and associated to the CAT tool project. The paths of the

translation memories corresponding with each domain are obtained and associated to the CAT tool project.

Screen of the CAT tool showing the dictionaries selected based on the domain values:

Screen of the CAT tool showing the translation memories based on the domain values:

DOMAIN IN THE CAT TOOL PROJECT EXPORT STEP

When the translation and proof‐reading tasks have ended in the CAT tool, the files are exported.

In this step, when the CAT tool project is closed the memory files are stored in the paths corresponding with

each domain.

The translated files advance in the L10N workflow to the post‐production phase.

File system showing the translation memories classified by domains:

25D3.2.2

LANGUAGE INFORMATION

LANGUAGE INFORMATION IN THE PRE‐PRODUCTION PHASE


If there is language information, it is checked if it is the same than the source language information declared in

the system (Its2Implementation.checkWfLangInfo()).

If not, a warning for the project manager is shown in the workflow.

Example:

LANGUAGE INFORMATION IN THE POST‐PRODUCTION PHASE


If the node has language information, update the value of the original language code with the target language

code (Its2Implementation.updateLocalLangInfo()).

The same process is carried out in the contents with HTML, but only within the parts that have been translated

(Its2Implementation.updateContentLangInfo()).

Example:

ALLOWED CHARACTERS

ALLOWED CHARACTERS IN THE POST‐PRODUCTION PHASE


If the allowed characters restriction is declared, it is checked with the regular expression

(Its2Implementation.checkLocalHtmlAllowedCharacters()).

If the restrictions are not fulfilled, an exception is raised, the process is aborted and the user is informed about

the reason.

Example:

<body id="36672" lang="de">

<body id="36672" lang="fr">

26D3.2.2

STORAGE SIZE

STORAGE SIZE IN THE PRE‐PRODUCTION PHASE


If the node is a translatable one and has declared storage size limitation, the maximum size is informed in the

attribute xMax of the translatable nodes (xlasTrad) of the CAT tool oriented xml. The size of the original

content is also stated (in the attribute xMaxOrig). This information will be available for the

translators/proof‐readers in the CAT tool. The size is calculated with the encoding.

(Its2Implementation.applyLocalStorageSizeRulesHtml()).

It is also checked if the original content fulfils the restriction. If not, a warning is shown to the project manager.

Example:

STORAGE SIZE IN THE POST‐PRODUCTION PHASE


If the node is a translatable one and has storage size limitation declared, a method checks the maximum

storage limitation compliance (Its2Implementation.checkLocalHtmlStorageSize()). It takes into account the

encoding declared for the content.

Example:

PROVENANCE

PROVENANCE IN THE PRE‐PRODUCTION PHASE


If there is provenance information available of a previous translation, the DB is updated to register the

translator and the language pair (Its2Implementation.applyWfProvenanceIntegration).

If there is provenance information available from a previous revision, the DB is updated to register the

reviser/proof‐reader and the language pair (Its2Implementation.applyWfProvenanceIntegration).

Example:

<div id="36672-node_title" its-allowed-characters="[^<>]">VDMA renforce la lutte contre le piratage des produits</div>

<div id="36672-node_title" its-storage-size="255">VDMA verstärkt den Kampf gegen Produktpiraterie</div>

<div id="36672-node_title" its-storage-size="255">VDMA renforce la lutte contre le piratage des produits</div>

<body id="36814" its-org="Linguaserve" its-person="21686" its-rev-org="Linguaserve" its-rev-person="20697">

27D3.2.2

PROVENANCE IN THE TRANSLATION CAT TOOL PHASE

The system suggests to the project manager the last translator who carried out the same task for the same

language pair. Page of CAT workflow ‐ Translation:

PROVENANCE IN THE REVISION CAT TOOL PHASE

The system suggests to the project manager the last reviser/proof‐reader who carried out the same task for

the same language pair. Page of CAT workflow ‐ Revision:

PROVENANCE IN THE POST‐PRODUCTION PHASE

The attributes related with the provenance information are updated

(Its2Implementation.updateLocalProvenance()). The translator (its‐person), the reviser or proof‐reader (its‐

rev‐person) and the organization (its‐org and its‐rev‐org) that have carried out the job.

Example:

READINESS (ITS 2.0 EXTENSION)

READINESS IN THE PRE‐PRODUCTION PHASE

A method obtains the ITS global rules (Its2Implementation.getItsGlobalRulesElement()).

Another method obtains the global readiness rules from the ITS global rules (Its2Implementation.

getGlobalReadinessRules()).

The expected delivery date is updated in the system, having into account the time zone, and the priority of the

translation is checked with the information available in the system

(Its2Implementation.applyWfReadinessIntegration()).

If there is no concordance, a warning for the technical department is shown in the workflow.

Example:

READINESS IN THE POST‐PRODUCTION PHASE

The date of availability for the next step in the chain is updated (attribute ready‐at) having into account the

time zone. Method Its2Implementation.updateGlobalReadiness().

<body id="36672" its-org="Linguaserve" its-person="21686" its-rev-org="Linguaserve" its-rev-person="20697">

<itsx:readinessRule ready-at="21/01/2013 13:48:56:000 CET" priority="1/3" complete-by="19/02/2013 16:00:00:000 CET" ready-to-process="hTranslate, reviseQA, hReview, publish"/>

28D3.2.2

The attribute with the processes to be carried out is updated (attribute ready‐to‐process), removing

completed tasks (human translation and revision for quality assurance). Method

Its2Implementation.updateGlobalReadiness().

If the file is processed after the expected delivery date, a warning for the project manager is shown in the

workflow.

Example:

<itsx:readinessRule complete-by="19/02/2013 16:00:00:000 CET" priority="1/3" ready-at="30/01/2013 17:46:27:744 CET" ready-to-process="hReview, publish"/>

29D3.2.2

9. APPEDIX II: ACCESS AND USER GUIDE

This appendix shows how to test, use and observe the results of the implementation of ITS 2.0. First, the CMS

side in Drupal by Cocomore and, in second place, the TMS side in Linguaserve’s environment.

9.1. CMS Side (Cocomore AG)

This section provides the links to demos, information access, and a guide on how to test, use and observe the

results of the implementation of ITS 2.0 in the CMS side in Cocomore’s environment.

Demo Page: Plastics and Rubber Machinery

URL: http://staging‐mlwlt.cocomore.com:8080/ (login: test , password: fenMarOf0)

User: tm

Password: tr4N5L4T1onm4N4G3r!

Tool: Drupal Module for editing and viewing of ITS 2.0 markup (Cocomore AG)

Source Code/Documentation: https://drupal.org/project/its

Tool: Drupal Module to connect to TMGMT Translator Linguaserve (Cocomore AG)

Description of the TMS side: Interchange between Content Management System and Translation

Management System

Source Code/Documentation: http://drupal.org/sandbox/kfritsche/1908422

Tool: Drupal Module to interact with TMGMT Workflow (Cocomore AG)

Details: Adds possibility to have additional steps before/after translation and integrates the Text

Analysis results from "Enrycher".

Source Code/Documentation: http://drupal.org/sandbox/kfritsche/1908598

Tool: ITS 2.0 jQuery Plugin (Cocomore AG)

Details: Selector plugin to read ITS 2.0 data from a node or select nodes by specified ITS markup.

Running software: http://plugins.jquery.com/its‐parser/

Source Code: https://github.com/attrib/jquery‐its2‐src

Documentation: https://github.com/attrib/jquery‐its2

If you are logged in you can see on top of each content page the links to edit the page, to access the Language

Management functionality described above, and to display the Translation Overview related to the respective

page.

30D3.2.2

In the edit menu you have the possibility to select content and add local ITS markup. At the bottom of the page

you can also set the domain and further ITS global data categories.

On the Language Management page you have the possibility to separately highlight different ITS data

categories in the content. If you select content, a menu appears at the selection to add ITS data directly. There

are also shortcuts for each data category to add categories fast and easy.

31D3.2.2

After the content is saved, you can go to the Translate page. There is an overview of all languages and the

status of translations for each language. At the left it is possible to select languages in which a translation is

needed. After the needed translations are selected, you can create a translation job by clicking on "Request

translation".

After a translation job is created you get an overview of all pages included in this job, the word count and the

current status. Here you can select a translator and set further settings, which the selected translator needs.

By clicking on submit the job, it is send to the translator and the translation process begins.

32D3.2.2

A second option is under Node Source. There it is possible to see all content pages and an overview of the

translation status. From this page you can create translation jobs with multiple pages at once or even select all

pages and translate them.

On the Job overview page you can see all translation jobs, the current status and the word count of this job.

Every ten minutes a cron automatically retrieves the status of all translation jobs from Linguaserve. If a

translation is finished, its status switches to "needs review" or "finished", depending on the settings you made

for the translator. If it is finished, the translation process is finished.

If you want to review it, you can set in the translator settings that every job needs reviewing and should not

be automatically published. To do this, just select the review button at the job overview and you’ll see the

screen like is shown below. You see the original content and the translated content side‐by‐side. Changes can

be carried out in the received translation. When the revision is made, click on the tick symbol for each item

and click on accept at the bottom. This creates the translated content page and sets the translation job on

finished status.

33D3.2.2

34D3.2.2

9.2. TMS side (Linguaserve)

This section provides the links to demos, information access and a guide about how to test, use and observe

the results of the implementation of ITS 2.0 in the TMS side in Linguaserve’s environment.

The system modules affected by the ITS 2.0 implementation are:

‐ Interchange of contents using the Global Business Connector Contents (GBCC) and webservices.

‐ Integration of Linguaserve’s internal localization workflow (PLINT).

‐ Development of a pre‐production/post‐production engine for Drupal XHTML files with ITS 2.0.

‐ CAT tool filter for CAT oriented XML files.

More detailed description about the process is available here: http://tinyurl.com/8woablr

Videos of the different processing steps are available here:

1) L10n workflow interaction for the pre‐production phase:

http://www.w3.org/International/multilingualweb/lt/wiki/images/6/67/Linguaserve_Preproduction_step_de

mo.zip

2) CAT tool usage with ITS 2.0:

http://www.w3.org/International/multilingualweb/lt/wiki/images/c/ca/Linguaserve_ITS_CAT_Tool_usage_de

mo.zip

3) L10n workflow interaction for the post‐production phase:

http://www.w3.org/International/multilingualweb/lt/wiki/images/a/aa/Linguaserve_Postproduction_step_de

mo.zip

Also a demonstration of how the pre‐production/post‐production engine for Drupal XHTML files with ITS 2.0 is

available in: https://www.linguaserve.net/las_demos/control/MLWLTWP3DemoEngine

user: demos // password: demosLingu@serve

Simple use guide:

a) To test the pre‐production step: Input files needed: Drupal XHTML files with ITS 2.0 (_orig.xml suffix).

Output files: CAT tool oriented XML file (_proc.xml suffix) and meta file (_meta.xml).

b) To test the post‐production step: Input files needed: CAT tool oriented XML file (_proc.xml suffix). Output

files: translated (*) Drupal XHTML files with ITS 2.0 (_trans.xml suffix).

(*) Translation simulation marks can be added to the content, which appear in the target language code

between asterisks.

The files highlighted in green and tagged as 'Example' are always available to try the demo.

193_demo_orig.xml is an original file in German (Drupal XHTML with ITS 2.0) and 193_demo_proc.xml is the

corresponding CAT tool oriented XML file translated in French. Feel free to download them and use them as

input for the demo.

The checkboxes with the ITS 2.0 data category names (Domain, Language Information…) are used to select the

information of the application of ITS desired to be displayed in the log area when processing files in the pre‐

production step. The log area shows the messages, warnings and errors, and the XPath of the nodes of the

processed file.

Documents

D3.2.2: B2B INTEGRATION SHOWCASE · The advantages of using an external ... localization workflow of the contents benefits from each ... ‐ DRUPAL is a free and