Quality Me as 2002

8/12/2019 Quality Me as 2002

1/31

The Measurement of Quality

in TranslationUsing Statistical Methods

The Measurement of Qualityhe Measurement of Quality

in Translationn TranslationUsing Statistical Methodssing Statistical Methods

43 rd ATA Conference Atlanta 200243 rd ATA Conference At lanta 2002

Riccardo Schiaffino President Aliquantum

Franco Zearo Worldwide Director, Language Services Lionbridge Technologies

2002 by J.D. Edwards and Lionbridge Technologies, Inc.

8/12/2019 Quality Me as 2002

2/31

Overviewverviewverview

Measuring QualityTranslation Quality AssessmentQuality Assurance Forms

Error CategoriesSampling

Translation Quality IndexQuestions and Answers

8/12/2019 Quality Me as 2002

3/31

You cannot measure qualityYou cannot measure qualityou cannot measure quality

This is not true:There are certain components of translationquality that will always remain subjective .

However ,There are other elements that can be objectivelymeasured.

By concentrating of these, we believe we canachieve a satisfactory measurement of translationquality.

8/12/2019 Quality Me as 2002

4/31

Why is Quality Measurement Important?hy is Quality Measurement Important?hy is Quality Measurement Important?

It is difficult to improve something if youcannot measure it.

Such measurement should berepeatable and objective .

Different evaluators should arrive atsimilar assessment for the same piece oftranslation.

8/12/2019 Quality Me as 2002

5/31

Who Benefits from Reliable TranslationQuality Measurement?

Who Benefits from Reliable Translationho Benefits from Reliable TranslationQuality Measurement?uality Measurement?

Professional Translators

Translation Companies and

In-House Translation Departments

Translation Customers and Users

8/12/2019 Quality Me as 2002

6/31

Our Definition of Qualityur Definition of Qualityur Definition of Quality

Functional approach to quality

Quality is defined as

consistently meetingthe needs and expectations

of the customer or user

8/12/2019 Quality Me as 2002

7/31

Product Process Assessmentroduct Process Assessmentroduct Process Assessment

Translation quality assessment must apply to

both :The translated text(the product)

The translation process(the process)

Aim:

Inputs Outcomes

Boundary Boundary

Feedback

Conversion Process

Process Process Process Process Process

8/12/2019 Quality Me as 2002

8/31

QC vs QACC vss QAA

Quality Control (QC)Quality verification over the whole text.Example: Editing.

Quality Assurance (QA)Sampling techniques, control of quality over a

(statistically significant) sample of the wholetext. Appropriate use: Quality measurement.

8/12/2019 Quality Me as 2002

9/31

Ideas from other disciplinesdeas from other disciplinesdeas from other disciplines

Software project management techniquesW. Edwards Deming and other qualityassurance experts

8/12/2019 Quality Me as 2002

10/31

Real Life Exampleseal Life Exampleseal Life Examples

Development of translation qualitymeasurement at J.D. EdwardsUse of sampling techniques for quality

assurance at Lionbridge

8/12/2019 Quality Me as 2002

11/31

Development of Translation QualityMeasurement at J D Edwards

Development of Translation Qualityevelopment of Translation QualityMeasurement at J D Edwardseasurement at J D Edwards

From the concept of checklists to aspreadsheet of measurementsChecklists are appropriate to control whether acertain action has been performed or not (e.g.,

spell check done or not as opposed to ameasurement of how many spelling mistakeswere found)

Based on LISA model (www.lisa.org)Flexibility (different settings for differentlanguages)

8/12/2019 Quality Me as 2002

12/31

Use of Quality Assurance Formsse of Quality Assurance Formsse of Quality Assurance Forms

The LISA Quality Assurance Form

Language: Reviewer: Date: Result: Pass Comments:

Client NameProject Name

Project Number Project Manager

Critical max. error points + 1Number of words 0 Major 5 pointsMax error points allowed 0 Minor 1 point

Error Category Minor Major Critical total max. allowed0 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

Total 0 0More elaborate descriptions of the error criteria can be found in the LISA QA model version 1.0 Reference Manual.

Quality Assurance Form

Consistency

Mistranslation AccuracyTerminologyLanguageStyleCountry

8/12/2019 Quality Me as 2002

13/31

The J.D. Edwards QA FormLanguage Customization

The J.D. Edwardshe J.D. Edwards QA FormA FormLanguage Customizationanguage Customization

Weighting the major categories

Accuracy 50%Style 15%Grammar 30%Formatting 5%

100%

1 - Give appropriate weight to the four following categories (total must add up to 10

Language Setup

Total

Categories

8/12/2019 Quality Me as 2002

14/31



Weighting the items within the major

categoriesIncorrect meaning 40%Non-standard terminolgy 20%Inconsistent terminolgy 20%Untranslated SL 20%

100%

Wrong register 40%Inappropriate anglicisms 30%Inappropriate use of passive/active voice 30%

100%

Spelling errors 20%Typos 15%Grammar errors 35%Syntax errors 25%Punctuation errors 5%

100%

Layout errors 50%Font errors 40%Double spaces 10%

100%

2 - Within the Accu ra cy category, give appropriate weight to the four following items (total must add up to 100%)

3 - Within the Style category, give appropriate weight to the three following items (total must add up to 100%)

4 - Within the Grammar category, give appropriate weight to the five following items (total must add up to 100%)

Total

Total

Accuracy

Style

Total

Formatting

Total

5 - Within the Formatting category, give appropriate weight to the five following items (total must add up to 100%)

Grammar

8/12/2019 Quality Me as 2002

15/31



Weighting the items within the major

categories (detail)

Incorrect meaning 40%

Non-standard terminolgy 20%Inconsistent terminolgy 20%Untranslated SL 20%

100%Total

Accuracy

8/12/2019 Quality Me as 2002

16/31

How We Worked to Develop OurSpreadsheet

How We Worked to Develop Ourow We Worked to Develop OurSpreadsheetpreadsheet

Determine type of errors, issues or problems

Determine relative importance of issues (may bedifferent for different languages; e.g., spellingerrors in English, French or Italian)

Determine which are the responsibility oftranslation

Determine tolerance limits for various levels ofquality

8/12/2019 Quality Me as 2002

17/31

Inspection Pointsnspection Pointsnspection Points

Key Principle: Reject defective material at its lowestKey Principle: Reject defective material at its lowestvaluevalue

QCQC

Proof Proof

Translation/EditTranslation/Edit

SourceSourceContentContentDevelopmentDevelopment(GIGO)(GIGO)

$ V a

l u e o

f S e r v

i c e

$ V a

l u e o

f S e r v

i c e

Stages of ProductionStages of Production

8/12/2019 Quality Me as 2002

18/31

Purposes of sampling according to LISAurposes of sampling according to LISAurposes of sampling according to LISA

To determine whether something has been

done or not.To accept / reject the batch of product at

hand.To determine if the process that producedthe product at hand was within acceptablelimits.

8/12/2019 Quality Me as 2002

19/31

Guidelines for Samplinguidelines for Samplinguidelines for Sampling

Select a sampleSelection criteria (e.g. random, systematic)Size considerationsCost considerations

Evaluate the sampleRepeatable, reproducible, objective

Investigate the outcome / causesCorrect / Improve

8/12/2019 Quality Me as 2002

20/31

Summary: Error Categorizationummary: Error Categorizationummary: Error Categorization

Select a (small) set of categories

CTQ: Critical-To-Quality categoriesProvide clear definitions

Set tolerance limitsMin / Max # of errors per X words

Assign a weight

Critical, Major, Minor

8/12/2019 Quality Me as 2002

21/31

Whats left to do ?hathat s left to do left to do ?

h f

8/12/2019 Quality Me as 2002

22/31

The concept of aTranslation Quality Index

The concept of ahe concept of aTranslation Quality Indexranslation Quality Index

Translation Quality Index (TQI)

A numberobtained by the rigorous

application of a QA formthat isindicative of the quality of a given

translation

8/12/2019 Quality Me as 2002

23/31

Index / Indicesndex / Indicesndex / Indices

Depending on ones purpose, there may

be more than a single TQI.E.g., a TQI may be developed for

external purposes (to standardize thework obtained from outsourcing).

Another TQI may be primarily for internalpurposes (to measure the quality of agiven special process).

l fA E l fE l f

8/12/2019 Quality Me as 2002

24/31

An Example of aTranslation Quality Index (1)

An Example of an Example of aTranslation Quality Indexranslation Quality Index (1)1)

LISA QA Model ver. 1.0 (1995)

3,000 words (12 pages @ 250 words)30 error points

30 error pts / 3,000 words = 1.0%

10,000 error pts out of 1 million wordsDPMO = 99.0% = TQI

A E l fA E l fE l f

8/12/2019 Quality Me as 2002

25/31



Microsoft Quality Standards for Print

ver. 1.0 (1998)10,000 words (40 pages @ 250 words)

0 major errors15 minor errors

15 errors / 10,000 words = 0.15%1,500 errors out of 1 million wordsDPMO = 99.85% = TQI

A E l fA E l fE l f

8/12/2019 Quality Me as 2002

26/31



2,000 words (8 pages @ 250 words)1 critical error 2 major errors3 minor errors

6 errors / 2,000 words = 0.3%

3,000 errors out of 1 million wordsDPMO = 99.7% = TQI

8/12/2019 Quality Me as 2002

27/31

Control Chartsontrol Chartsontrol Charts

Concept of statistical control

98.50

98.75

99.00

99.25

TQI Control Chart

C2

C 3

C4

C 5

C 6

C7

C 8

C 9

C1 0

C1 1

C1 2

C1 3

UCL LCL Mean Data Point

8/12/2019 Quality Me as 2002

28/31

Process Flow Diagramrocess Flow Diagramrocess Flow Diagram

Aim:

Inputs Outcomes

Boundary Boundary

Feedback

Conversion Process

Process Process Process Process Process

How to Apply Statistical Methods for QualityHow to Apply Statistical Methods for Qualityow to Apply Statistical Methods for Quality

8/12/2019 Quality Me as 2002

29/31

How to Apply Statistical Methods for QualityImprovement

How to Apply Statistical Methods for Qualityow to Apply Statistical Methods for QualityImprovementmprovement

1. Define error categories and tolerances

2. Create a QA form

3. Obtain a TQI index4. Use the TQI index to improve the

translation process

8/12/2019 Quality Me as 2002

30/31

ContactsContacts

[email protected] Aliquantum, Inc.

[email protected] Lionbridge Technologies, Inc.

For more information about translation issues, visit

TranslationQuality.com

Biographical Notes on the Authorsiographical Notes on the Authorsiographical Notes on the Authors

8/12/2019 Quality Me as 2002

31/31

Biographical Notes on the Authorsiographical Notes on the Authorsiographical Notes on the Authors

Riccardo SchiaffinoRiccardo Schiaffino worked as translator, translation manager and special software translationproject lead for a major software company, and now leads a small company he established with a

few experienced colleagues. As a translation manager, Riccardo worked on the improvement oftranslation quality and on translation quality metrics and tools. He holds an MA degree inTranslation, and has been working in translations for over 20 years, first in Italy and then in theU.S. Riccardo is ATA accredited.

Franco Pietro ZearoFranco Pietro Zearo is a project manager with Lionbridge Technologies in Boulder, Colorado. Heholds a degree in translation from the Advanced School of Modern Languages for Translators andInterpreters at the University of Trieste, Italy, and earned an MBA from the University of Phoenix.Before joining Lionbridge in 1996, he worked as a freelance technical translator in Italian, English,and Russian. At Lionbridge, he has held positions in translation, localization analysis, presales,and cultural and globalization consulting. He has been responsible for translation quality onnumerous projects for many Fortune 500 clients. In his previous role as senior technical translator,he helped define best practices for the translation department.

Documents

Quality Me as 2002