14
WORK IN PROGRESS PLEASE DO NOT CITE 1 Questionnaire translation in the Third European Company Survey. Conditions conducive for the effective implementation of the TRAPD approach Dr Maurizio Curtarelli (Eurofound) 1 Dr Gijs Van Houten (Eurofound) 2 Abstract: The European Foundation for the Improvement of Living and Working Conditions (Eurofound) carries out three recurring Europe-wide surveys: the European Working Conditions Survey (EWCS), the European Quality of Life Survey (EQLS), and the European Company Survey (ECS). In the latest wave of the ECS - an establishment level telephone survey, which is carried out in 32 countries and in 36 languages or language versions - Eurofound for the first time followed the Translation-Review-Adjudication-Pretesting-Documenting (TRAPD) approach to survey translation. The survey consists of two questionnaires: one for the manager in charge of human resources and one for an official employee representative. The questionnaires were initially translated into German and French, after which the English source questionnaire was reviewed and translation instructions were drafted. The questionnaires were pretested in three countries, using cognitive and structured interviews. After implementing changes based on the pre-test findings, the questionnaires were translated into all 36 languages or language versions. For each language version, the translation was carried out by two independent translators after which an adjudication process took place, led by a third person. In countries where different versions of the same language are used (e.g. Germany and Austria) a cross-national adjudication process was put in place between the country level adjudicators to track unwarranted differences between the language versions. This paper will further outline the approach and will focus on the conditions that proved particularly conducive or obstructive for its efficient and effective implementation. Introduction Questionnaire translation is a crucial aspect in cross-country survey research, as it is directly concerned with collecting comparable data in different countries or among different lingual groups. In the widest sense, translation aims to ensure the implementation of ‘equivalent’ instruments, in different lingual, sociocultural and institutional settings. However, the term ‘appropriate’ and ‘adequate’ should be preferred with reference to questionnaire translation in survey research, as ‘researchers in both translatology and in survey-based research disagree on the suitability and scope of “equivalence” vocabulary and frameworks’ (Harkness et alia, 2010, p.119). The underpinning idea is that a good questionnaire translation should be not just the simple and systematic transposition of questionnaire items from the source language (‘source questionnaire’) into the target language 3 (‘target questionnaire’), following one of the numerous approaches within the translation theory (e.g. transparent or covert translations, word-for-word translation, literal translation, close translation, etc.). A good questionnaire translationshould, on the one hand,take into consideration the different social realities, cultural norms, and respondent needs (e.g. level of vocabulary) existing within and across countries, adapting translated questions accordingly, and,on the other hand, respect the questionnaire design (e.g. using the same scales). In Harkness et alia words, a successful survey translation is expected to do all of the following: keep the content of the questions semantically similar; within the bounds of the target language, keep the question format similar; retain measurement properties, including the range of response options offered; and maintain the same stimulus. Such matters as burden and form of disclosure are also meant to be kept constant. The question design stage determines whether most of these have any chance of being realized in translation’ (Harkness et alia, 2010; p. 117). 1 Working Conditions and Industrial Relations Research Unit.Email: [email protected] 2 Working Conditions and Industrial Relations Research Unit. Email: [email protected] 3 The ‘source language’ should be intended as the language translated out of, while the ‘target language’ is the language translated into.

Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

Embed Size (px)

Citation preview

Page 1: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

1

Questionnaire translation in the Third European Company Survey. Conditions

conducive for the effective implementation of the TRAPD approach

Dr Maurizio Curtarelli (Eurofound)1

Dr Gijs Van Houten (Eurofound)2

Abstract:

The European Foundation for the Improvement of Living and Working Conditions (Eurofound) carries out three recurring Europe-wide

surveys: the European Working Conditions Survey (EWCS), the European Quality of Life Survey (EQLS), and the European Company

Survey (ECS). In the latest wave of the ECS - an establishment level telephone survey, which is carried out in 32 countries and in 36 languages or language versions - Eurofound for the first time followed the Translation-Review-Adjudication-Pretesting-Documenting

(TRAPD) approach to survey translation. The survey consists of two questionnaires: one for the manager in charge of human resources and

one for an official employee representative. The questionnaires were initially translated into German and French, after which the English source questionnaire was reviewed and translation instructions were drafted. The questionnaires were pretested in three countries, using

cognitive and structured interviews. After implementing changes based on the pre-test findings, the questionnaires were translated into all 36

languages or language versions. For each language version, the translation was carried out by two independent translators after which an adjudication process took place, led by a third person. In countries where different versions of the same language are used (e.g. Germany and

Austria) a cross-national adjudication process was put in place between the country level adjudicators to track unwarranted differences

between the language versions. This paper will further outline the approach and will focus on the conditions that proved particularly conducive or obstructive for its efficient and effective implementation.

Introduction

Questionnaire translation is a crucial aspect in cross-country survey research, as it is directly

concerned with collecting comparable data in different countries or among different lingual

groups. In the widest sense, translation aims to ensure the implementation of ‘equivalent’

instruments, in different lingual, sociocultural and institutional settings. However, the term

‘appropriate’ and ‘adequate’ should be preferred with reference to questionnaire translation in

survey research, as ‘researchers in both translatology and in survey-based research disagree on

the suitability and scope of “equivalence” vocabulary and frameworks’ (Harkness et alia,

2010, p.119).

The underpinning idea is that a good questionnaire translation should be not just the simple

and systematic transposition of questionnaire items from the source language (‘source

questionnaire’) into the target language3(‘target questionnaire’), following one of the

numerous approaches within the translation theory (e.g. transparent or covert translations,

word-for-word translation, literal translation, close translation, etc.).

A good questionnaire translationshould, on the one hand,take into consideration the different

social realities, cultural norms, and respondent needs (e.g. level of vocabulary) existing within

and across countries, adapting translated questions accordingly, and,on the other hand, respect

the questionnaire design (e.g. using the same scales). In Harkness et alia words, ‘a successful

survey translation is expected to do all of the following: keep the content of the questions

semantically similar; within the bounds of the target language, keep the question format

similar; retain measurement properties, including the range of response options offered; and

maintain the same stimulus. Such matters as burden and form of disclosure are also meant to

be kept constant. The question design stage determines whether most of these have any

chance of being realized in translation’ (Harkness et alia, 2010; p. 117).

1Working Conditions and Industrial Relations Research Unit.Email: [email protected]

2Working Conditions and Industrial Relations Research Unit. Email: [email protected]

3The ‘source language’ should be intended as the language translated out of, while the ‘target language’ is the language

translated into.

Page 2: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

2

In order to ensure expected high quality standards of translations, aquestionnaire translation

quality framework and related assessment procedure in terms of this framework should be

designed and implemented. However, due to the fact that ‘there continues to be a disjoint

between theories, practice, and benchmarks acknowledged in the admittedly diverse

translation sciences and the various approaches taken to translations and to assessment of

translation in survey research’ (Harkness et alia, 2010: p.118), the ‘“established” translation

practice in survey research, in the sense of what is commonly done, is […] by no means good

practice’ (Harkness et alia, 2010: p.118).

Among the proceduresto assess questionnaire translation quality, the ‘back translation’(BT)

procedureappears to be the most commonly associated to survey translation, in spite of very

little research on the effectiveness of this assessment procedure compared to other forms of

translation assessment, and, by contrast, of growing research criticizing the BT procedure, and

indicating that other forms of translation review and assessment are more efficient than BT. In

its simplest form, the BT procedure implies that the translation produced for a target language

population is back-translated into the source language. The two source language versions are

then compared to try to find out if there are problems in the target language text. However,

comparisons of an original source questionnaire and a back-translated source questionnaire

provide only limited and potentially misleading insight into the quality of the target language

version of the questionnaire.

As Harkness defended, instead of looking at two source language texts, it is more valuable in

practical and theoretical terms to focus attention on first producing the best possible

translation and then directly assessing the translation produced in the target language, rather

than indirectly through a BT procedure.

For all of the above, Harkness described the framework for a five-step team-based translation

procedure that involves multiple levels of review and reconciliation known as Translate,

Review, Adjudicate, Pretest, and Document (TRAPD). The TRAPD framework encompasses

a multifaceted view, emphasizing both linguistic and sociocultural elements. The TRAPD

model is illustrated in Figure 1.

Figure 1. The TRAPD team-based translation model

Source: Harkness, 2011

Page 3: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

3

The TRAPD model can be shortly illustrated as follows.

Step 1: Translation

The source questionnaire is translated by two different individual translators or two different

teams of translators. Translators produce two draft versions of questionnaire translations, and

attend review meetings.

Step 2: Review

Experts review the initial translations and suggest modifications based on their judgment and

expertise. Review meetings with translators can be useful to clarify issues.

Step 3: Adjudication

The adjudicator makes decisions to reconcile options from the preceding two steps and

provide a final version of the target questionnaire to be tested.

Step 4: Pretesting

Cognitive interviews to identify difficulties in understanding and answering the questions and

to identify problems that could impede comparability are carried out. Questionnaire is

pretested to identify further problems with translated versions. Suggestions for modifications

to the target questionnaire can arise in this step.

Transversal activity: Documentation

Qualitative and quantitative data (e.g. notes, minutes form meetings, etc.) should be collected

at each stage of the process and made available for supporting the work of people involved in

the process (translators, adjudicators; e.g. for questionnaire modification and improvement)

and for assessment purposes.

In consideration of its numerous advantages in terms of translation quality and quality

assessment, especially when compared to BT procedure, TRAPD approach is becoming more

and more popular in survey research, possibly also because ‘good translations do not ensure a

good survey but bad translations do guarantee a bad survey’ (Harkness, 2010: p. 129).

Also the European Foundation for the Improvement of Living and Working conditions

(Eurofound), which conducts three European cross-national periodical surveys, decided to

adopt the TRAPD approach for the first time in occasion of the recent last wave of its

European Company Survey (ECS).

The aim of this short paper is therefore to provide a reflection on the TRAPD translation

procedure related to the Third European Company Survey. An overview of Eurofound surveys

(paragraph 1) precedes an accurate description of the ECS questionnaire translation procedure

and quality assessment framework (paragraph 2). Criticisms, lessons from the experience and

indication for future improvement conclude the paper (paragraph 3).

Page 4: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

4

1. The translation process in Eurofound Surveys

Eurofound surveys

The European Foundation for the Improvement of Living and Working Conditions

(Eurofound) is a European Agency, governed by a tripartite Governing Board. Eurofound

carries out three repeated cross-sectional surveys to fulfil its mission to provide high quality

information and advice to its stakeholders.4

The European Working Conditions Survey (EWCS) is the longest running survey. It was first

implemented in 1990 and has so far been carried out five times; a sixth wave is to be fielded

in 2015. Themes covered include employment status, working time arrangements, work

organisation, learning and training, physical and psychosocial risk factors, health and safety,

worker participation, work-life balance, earnings and financial security, as well as work and

health. The European Quality of Life Survey (EQLS), implemented in 2003, 2007 and 2011,

contains a broad range of indicators on different dimensions of quality of life, both objective

and subjective. Finally, the European Company Survey (ECS), implemented in 2004, 2009

and now in 2013, gives an overview of workplace practices and how they are negotiated in

European establishments. It is based on the telephone interviews with both managers and

employee representatives.

The three Eurofound surveys share the following features:

- Random probability samples

- Coverage of all EU Member States as well as (potential) candidate countries and

EFTA countries

- Questionnaires covering a wide range of topics designed to meet European policy

needs, built on an inter-disciplinary scientific basis

- Developed in close cooperation with Eurofound’s tripartite stakeholders and experts

in the field

- Elaborate strategy for quality assurance

- Timely availability of results, in reports, an online data processing tool, and

bypublishing the micro-data on the UK Data Archive.

Organisational structure

Eurofound outlines the survey design and the quality assurance strategy and is responsible for

developing the questionnaire, but the preparation and implementation of fieldwork – including

most of the work on questionnaire translation - is contracted out. The main contractor is a

coordinating centre that coordinates a network of national fieldwork agencies in all the

countries covered by the survey. Some of these fieldwork agencies would be part of the same

company as the coordinating centre and some of them would be sub-contractors.

Consequently, national fieldwork agencies differ, amongst other things, in (1) their ‘usual’

approach to survey implementation, (2) their level of commitment to the contract, and (3) the

extent to which they can be controlled by the coordinating centre.

This organisational structure has implications for the decisions in the survey design to

harmonise on the one hand, and to opt for functional equivalence (e.g. allow for local

variation) on the other hand. Mismatches between what is prescribed centrally and local

common practice bear the risk of (1) increased cost, and (2) reduced. In writing tender

specifications and designing a quality assurance strategy Eurofound therefore has to balance

4 See http://www.eurofound.europa.eu/surveys/index.htm.

Page 5: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

5

between prescribing what they consider ‘best practice’ and requesting tenderers to suggest

what works best in each of the countries.

Translation of Eurofound surveys

Eurofound’s approach to translation has evolved from completely outsourcing translation to

translation agencies to a collaborative effort between the fieldwork contractor, Eurofound staff

and experts from Eurofound’s network of observatories. The following process was applied in

the most recent waves of the EWCS, EQLS and the first and second ECS:

1. For each target language, two native speakers (proficient in English) each performed an

independent translation of the master questionnaire.

2. A third translator then combined the two versions into one.

3. A fourth translator (not familiar with the material) then translated the questionnaire back

into

English (back-translation).

4. The results of the back-translation were reviewed centrally.

5. Finally, the fieldwork contractor’s coordinating centre approved all final target-language

translations.

The role of the outside experts varied across the surveys. For the 5th EWCS Eurofound’s

correspondents from Eurofound’s European Working Conditions Observatory (EWCO)

validated the translations for each language used in the EU27 and Norway.5 These

correspondents – experts in working conditions at national level – were responsible for checking

the questionnaire in their native language (and also in most of the minority languages in their

countries). It was the results of this validation exercise that triggered a rethinking within

Eurofound about the translation procedure that was applied, which eventually lead to the

embracing of the TRAPD approach for the translation of the questionnaire for the third

European Company Survey.

5See http://www.eurofound.europa.eu/surveys/ewcs/2010/documents/translationprocess.pdf.

Page 6: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

6

2. The TRADP approach in the Third European Company Survey

The European Company Survey (ECS) has been carried out every four years by Eurofound

since its inception in 2004-2005. The second survey was completed in 2009 and the third

survey took place in the first six months of 2013.The survey has the following objectives:

to map, assess and quantify information on company policies and practices across Europe

on an harmonised basis;

to analyse relationships between company practices and their impact as well as looking at

practices from the point of view of structures at company level, focusing in particular on

social dialogue;

to allow for the development of homogeneous indicators on these issues in order to

monitor trends at European level.

The ECS is a questionnaire-based representative sample survey carried out by telephone in the

language(s) of the country.

One of the special features of the survey is that interviews take place with the manager

responsible for human resources in the establishment and when possible with an employee

representative. That means that two different questionnaires are developed and used to carry

out the survey.

Another specificity of the survey is that each round covers a different topic, which is

investigated through specific questions, while a number of indicators are kept constant from

round to round in order to ensure comparability over the time. The first wave of the survey

covered issues around working time arrangements and work-life balance at company level.

The second wave looked at different forms of flexibility, including working-time flexibility,

contractual flexibility, variable pay and financial participation, as well as accompanying

human resource measures, and the nature and quality of workplace social dialogue. The

thirdwavehas looked at workplace organisation, workplace innovation, employee participation

and social dialogue in European workplaces.

The sample is random and representative of the establishments in the countries, stratified by

sector and establishment size. The sample size for the 3rd

ECS has been of minimum 500

interviews for the smallest countries (with the exception of Malta, FYROM, Iceland,

Montenegro and Turkey, with 300 interviews) up to 1,650 in the biggest (UK, Poland, Italy,

France, Spain, Germany) with a total of 30,113 interviews.

The geographical scope of the survey has expanded over time:

the First ECS(2004-2005) covered 21 countries, 15 ‘old’ EU Member States plus Cyprus,

the Czech Republic, Hungary, Latvia, Poland and Slovenia;

the Second ECS (2009) covered 30 countries including the 27 EU Member States,

Croatia, the Former Yugoslav Republic of Macedonia (FYROM) and Turkey;

the3rd

ECS (2013) covered 32 countries including 27 EU Member States and Croatia, the

Former Yugoslav Republic of Macedonia (FYROM), Iceland, Montenegro and Turkey.

In consideration of thegeographical scope of the survey, the source questionnaires (drafted in

English) have been translated into 36target languages, including the key minority or co-

official languages spoken in the surveyed countries. For another eight language variants, an

adaptation was made, based on an original translation carried out in another country (see

Table 1).

Page 7: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

7

Table 1: List of target languages and related countries for the Third European Company

Survey TRAPD translation questionnaires procedure

Country Language Original translation / Adaptation

Belgium (BE) Dutch Original

French Original

Bulgaria (BG) Bulgarian Original

Czech Republic (CZ) Czech Original

Denmark (DK) Danish Original

Germany (DE) German Original

Estonia (EE) Estonian Original

Russian Original

Greece (EL) Greek Original

Spain (ES) Spanish (Castilian) Original

Catalan Original

France (FR) French Original

Ireland (IE) English Adaptation from English source

Italy (IT) Italian Original

Cyprus (CY) Greek Original

Latvia (LV) Latvian Original

Russian Adaptation from EE-RU version

Lithuania (LT) Lithuanian Original

Russian Adaptation from EE-RU version

Luxembourg (LU)

French Adaptation from French version

German Adaptation from German version

Luxemburgish Original

Hungary (HU) Hungarian Original

Malta (MT) Maltese Original

English Adaptation from English source

Netherlands (NL) Dutch Original

Austria (AT) German Original

Poland (PL) Polish Original

Portugal (PT) Portuguese Original

Romania (RO) Romanian Original

Slovenia (SI) Slovene Original

Slovakia (SK) Slovak Original

Hungarian Adaptation from HU version

Finland (FI) Finnish Original

Swedish Adaptation from Swedish version

Sweden (SE) Swedish Original

United Kingdom (UK) English Original

Croatia (HR) Croatian Original

Republic of Macedonia-

(MK) Macedonian (MKD) Original

Albanian (ALB) Original

Montenegro (MON) Montenegrin Original

Page 8: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

8

Country Language Original translation / Adaptation

Serbian Original

Iceland (IS) Icelandic Original

Turkey (TR) Turkish Original

Source: Eurofound, 2013

The translation exercise for the 3rd

ECS questionnaires has represented a noticeable effort, in

consideration on the one hand of the number of language versions needed for the purposes of

the survey, and on the other hand for the quality standards that Eurofound is committed to

achieve for its mandate and role within the EU bodies. For such reasons Eurofound decided to

adopt for the first time the TRAPD procedure for one of its surveys. However, the procedure

has been adapted to Eurofound specific needs, considering time and budget constraints.

The result is a multi-step procedure illustrated in Figure 2 which can be described as follows.

Step 1: Translation and pre-test of the source questionnaires

The source questionnaires were translated by two professional translators into German and

Frenchwith the aim of detecting difficulties when translating from English into other

languages. The two target questionnaires and the source questionnaires were subsequently

pre-tested in Germany, France and Ireland through cognitive interviews to identify difficulties

in understanding and answering the questions and to identify problems that could impede

comparability. The results of this process and comments from the translators helped in

reviewing the questionnaires and to get a final draft source version of the questionnaires.

Step 2: Translation

The source draft questionnaires were translated item-by-item by two different individual

translators using a specific software (WebTrans), which allows recording the translated items

and also any concerns, questions or comments when translating each item. WebTrans

automatically ensured that items included multiple times in the questionnaire (i.e. identical

response scales, or the same expressions) appeared only once, so that these items were

translated identically.

Step 3: Adjudication/review

Based on the two translations, and considering the notes prepared by the translators in the

previous step, a third, synthesized version was created in collaboration with the two translators

and the adjudicator. The translators and the adjudicator participated in a “review meeting” to

decide on questions where the two translations were not in agreement. The adjudicator was

ultimately responsible for the final target questionnaires.These meetings involved the

consideration of the definition of the original term and attempt to agree on a target language

wording that was the most relevant translation.

Step 4: Harmonisation

The questionnaires were translated into the target language as many times as many countries

had the particular target language. In order to maintain coherence across country variants of

the questionnaires in the same language, national adjudicators, once their “final” translation

was completed, were required to share their national translations, and discuss possibilities of

harmonisation with like-language countries’ adjudicators. This process was decentralised, and

took place online via a specific software for meetings online (Webex).

Step 5: Review and validation

The documentation for all final target questionnaires was checked by Eurofound staff. For

Page 9: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

9

those languages where native speakers were available among Eurofound staff, especially for

those where the documentation was considered not satisfactory, the final target questionnaires

themselves were quality checked.Adjudicators were subsequently required to modify the

target questionnaire or provide a better documentation, as appropriate.

Step 6: Pilot interviews

Validated target questionnaires were finally tested as part of the pilot interviews that were

carried out to test the survey instrument on the first 50 cases in each country. The testing of

the questionnaire focused on grammatical errors, typos and formulations that were difficult to

read to or understand by respondents. Final edits in the target questionnaires were made on the

basis of the interviewers’ feedback, and proposed and approved by Eurofound.

Transversal activity: Documentation

All the steps of the translation procedure have been carefully documented in order to spot

issues and understand the reasons for some specific choices. Documentation produced at each

stage of the process has been used to support the following step and to modify/improve the

target questionnaires. In particular, the steps where documentation is crucial have been the

adjudication/review and the harmonization steps. For the adjudication/review step, comments

were provided by the team for each question and item, including how a final solution was

reached and why one option was preferred over another. These comments were stored in

WebTrans. Regarding the harmonization step, the final modifications that were based on the

cross-country harmonisation meetings were also commented on in the WebTrans database,

with clear indication that the change was the result of this effort.

Some more specific features of the TRAPD procedure applied to the Third European

Company Survey have been the following.

1. Local translation teams/central coordination

The translation teams in each country were composed of three individuals: two persons who

independently performed forward translations of the instruments, and an adjudicator. These

three individuals were responsible for participating in an online meeting to discuss items in

the questionnaire and to aim to come to an agreement on these items. Based on these

discussions, the adjudicator was ultimately responsible for merging the two translations into a

single national version of the questionnaire. The persons selected for this task were selected

among persons who routinely perform questionnaire translations – and of course had excellent

command of the source language (English) and were native speakers of the target language.

The whole translation process was coordinated centrally by the contractor in each step, with

the constant support and advise of Eurofound staff.

2. Training of translators and adjudicators

Prior to starting their assignments, each member of the national translation teams received an

interactive training about the questionnaires and the tasks to be conducted. These trainings

were conducted in English via Webex videoconferencing system and were recorded for

quality assurance and reference purposes. All translators and adjudicators were scheduled for

training according to language groups and were well prepared beforehand with manuals and

information on the questionnaire to help facilitate their learning in the training.

3. Provision of supporting material

The translators and adjudicators were provided with reference material to support and guide

their work. In particular, they were provided with a glossary in local language and specific

written guidelines. The glossary held explanations for the translation teams to support the

functionally equivalent translation of the key terms used in the survey instruments.

Page 10: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

10

Eurofound’s EIRO experts created short target-language vocabularies, offering translations for

some crucial terms, which were recommended to be used. The guidelines for translators and

adjudicators had the aim of giving advise on how to deal with issues e.g. related to

appropriateness of technical terms, precision of translation, meaning of words, use of tenses

and verbs, etc.

4. Use of specific software packages

The translation and adjudication process have been supported by the use of specific software,

called ‘Webtrans’ which has proved to be a reliable tool for conducting the translation process

for a large number of multi-country / multi-language survey. It was extended with an added

feature (‘workshop mode’) specifically to facilitate and document the adjudication/review

team meetings, allowing the easy identification of items for translators to discuss. The

advantage of such a software was the possibility of recording and documenting all the activity

on the platform. Also the training of translators and adjudicators was supported by a specific

platform (WebEx). Some of the benefits in using Webex include being able to record all

training sessions and also have the ability to use Skype and telephoning to join the

conference/training when there were rare circumstances where calling through the computer

proved problematic.

Page 11: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

11

Figure 2. The TRAPD team-based translation model applied to the Third European Company Survey

Source: Eurofound, 2013

Target questionnaires (36 (+ 8) languages)

Source questionnaires (English)

German translation French translation

Pre-test: cognitive interviews in three languages

Source questionnaires (English)

Translation 1 Translation 2

Pilot interviews in each country

Review

Fieldwork

Training Guidelines Glossary D

O C

U M

E N

T A

T I

O N

Adjudication/review

Cross-country harmonisation

Page 12: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

12

3. Lessons from the experience

The collaborative translation approach adopted by the ECS was a first time for Eurofound as well

as for the fieldwork contractor and their national partners. The translation procedure introduced

for the 3rd

ECS differed quite significantly from the process to which the fieldwork contractor

and their national partners were accustomed. For the 3rd

ECS a team-based setup was adopted,

whereas the contractor’s usual translation scheme involved two forward translations reconciled

by the next hierarchy level, verified by back-translations. Also, independent translations were

produced for the same target languages if used in different countries whereas the contractor

would usually rely on adapting a single translation to country variants in order to maintain

consistency in the wording across the different countries.

The outcome of the exercise was not fully satisfactory for a number of the target languages and

required an extra effort to bring the corresponding target questionnaires to the expected quality

standard. The review and validation process (Step 5) by Eurofound staff highlighted in fact two

main issues.

The first issue consisted in that some national teams failed to fully document their activities and

to adequately illustrate the reasoning behind their translation choices, which make difficult to

assess properly the quality of the process as the TRAPD approach supposes. These cases of

unsatisfactory documentation brought to a request for further and more extensive documentation

in order to validate and approve the final version or, whenever possible, were compensated by a

thoroughly revision of the questionnaire by Eurofound staff.

The second issue concerned the translations’ quality itself, to the extent to which Eurofound staff

could assess it (not all the target languages are spoken by Eurofound staff). In the case of some

national teams, the translation produced was of very good quality, while some others national

teams produced translations of lower quality and required to be revised by the corresponding

national team.

This outcome stemmed from the composition and level of commitment and motivation (beyond

the pecuniary aspects) of the national teams, and can be considered as a consequence of the lack

of the scientific infrastructure that supports other cross-national surveys (e.g. the ESS) and which

allows the ‘task ownership’ and intimate familiarity with the measurement tool required by the

TRAPD approach.

The implementation of a collaborative, iterative questionnaire “development” in each target

language by a network of commercial providers - to a large extent the same normally employed

by the contractor following the BT procedure - proved to be difficult, particularly because

(regardless of variations to the level of commitment to quality) commercial organisations by

definition are geared more toward effectiveness than toward scientific excellence.

Also, the sufficient skill level among the members of the translation teams in terms of language

proficiency and previous translation experience of the translators - included in the tender

specifications to ensure a formal quality criterion - was not always fully reflected in the skill

level of some national teams. The same applied for the IT skills required to use the translation

software (WebTrans) in a proper and fruitful way.

It has to be pointed out that, in order to achieve the ‘task ownership’, motivate the national

teams, and familiarize them with the tasks to be developed within the translation procedure, an

elaborate training process was put in place. However, it can be said that it did not fully achieved

its objectives, possibly due to the short duration of the training session (around half an hour), and

the online rather than face-to-face training.

To conclude, it is worth to indicate some possible improvements.

Page 13: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

13

First, in order to achieve the ‘task ownership’ and the required level of commitment, a rethinking

of the composition and rules of engagement of national team members is needed, preferring

translators with a ‘scientific’ profile to those with a commercial one or delegating experts in the

translation teams, directly.

Second, the training is important as an opportunity for the team members to familiarise with the

survey and the purposes and method of the translation procedure, to create a sense of belonging

to the national team, increase motivation and enhance sense of commitment, but it should be

extensive, covering carefully all the aspects of the survey, the requirements in terms of outcomes

and documentation. The training should be possibly face-to-face to give the opportunity to the

team members of knowing each other better.

Finally, more time should be devoted to the actual translation activity, so that translators have

enough time to thoroughly complete their individual roles, enabling more effect in later phases of

translation as the ones of reflection, discussion and review.

Page 14: Questionnaire translation in the 3rd European Company · PDF fileQuestionnaire translation in the Third European ... preferred with reference to questionnaire translation in ... cross-national

WORK IN PROGRESS – PLEASE DO NOT CITE

14

References

Eurofound (2013), Third European Company Survey (COMPLETE)

Harkness, J. (2011), Translation, in Cross-cultural surveys guidelines, Comparative Survey

Design and Implementation (CSDI) Guidelines Initiative,

http://ccsg.isr.umich.edu/translation.cfm

Harkness J., et alia, (2010), Translation, adaptation and design, in Harkness J., et alia (Eds.)

(2010), Survey Methods in Multination, Multiregional, and Multicultural Contexts, Wiley &

Sons, New Jersey.