THE IMPORTANCE OF CONCEPT ELABORATION GUIDES IN LINGUISTIC VALIDATION AND

RECOMMENDATIONS FOR DEVELOPMENTBarbara Brandt, MA1; Tim Poepsel, PhD2; Elizabeth Yohe Moore, MPH2; Shawn McKown, MA1; Elizabeth McCullough, MA1

1RWS Life Sciences, Hartford, CT, USA; 2RWS Life Sciences, Chicago, IL, USA

AIMS

The objective of this study was to review and analyze Concept Elaboration Guides (CEGs) created for Clinical Outcomes Assessments (COAs), and then identify the most beneficial approaches for building CEGs from a localization and cognitive interview perspective.

BACKGROUND

CEGs, which are sometimes called “Item Description Guides” or “Item Definition Guides,” provide important guidance for linguists and interviewers during the linguistic validation process. This document is created by the instrument developer and provided to the Language Service Provider (LSP) or created by a member of the LSP linguistic validation team prior to initial forward translation of the COA. CEGs serve a dual purpose of both guiding the linguists during translation and harmonization (see ‘Linguistic Validation’ section below) and providing insight to cognitive interviewers about the intent of each item. During translation, linguists should have access to conceptually equivalent alternative terminology in the event that the source text needs localization in order to achieve a culturally appropriate and accessible translation. As is the case with many COAs that assess culturally specific activities of daily living (such as household tasks, games or sports) additional consideration is needed during translation to adapt activities for the target country while maintaining conceptual equivalence [1]. During cognitive debriefing, interviewers should have information about the intent of each COA item as well as elaboration of medical and technical terminology, in order to accurately evaluate interview subject comprehension.

LINGUISTIC VALIDATION

Linguistic Validation is a process conducted to confirm that a Clinical Outcomes Assessment (COA) questionnaire is acceptable for use in different languages and in different cultural contexts, aligned with the ISPOR Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures [2]. Without this careful development of a translation and subsequent cognitive debriefing, one cannot be reasonably certain that the adapted instrument is both conceptually equivalent to the original and clearly understood by the average patient. The linguistic validation process begins with two translators independently translating the instrument into the target language. The translators then exchange drafts and work together to develop one reconciled or “harmonized” version. At that point, the harmonized translation is provided to a third translator who translates the text back into English without access to the original English. Both the harmonized translation and the English back translation are reviewed by a project manager and a survey research analyst, and adaptations to the translation are made as needed. Once the final translation has been approved, it is debriefed with a sample of in-country native speakers of the language, with varying demographic and educational backgrounds, to check for conceptual equivalence and clarity.

METHODS

30 CEGs created for linguistic validation projects involving translation, harmonization and cognitive debriefing interviews were selected for review using convenience sampling of the RWS Life Sciences CEG archive. CEGs eligible for selection for this review were limited to those utilized for RWS Life Sciences linguistic validation projects completed from 2013-2018. A survey research analyst reviewed each CEG. During the course of the review, the elements of the CEG were identified that are most critical in assisting linguists with translation and harmonization, and interviewers with cognitive debriefing.

RESULTS

Of the 30 CEGs selected, 23% (7) were provided by the questionnaire developer prior to translation, 20% (6) were created by RWS and then reviewed and revised by the developer prior to translation, and 57% (17) were created by RWS alone with no input from the developer. 53% (16) of the CEGs selected were developed for health-related quality of life questionnaires that can be used to evaluate a wide range of health conditions and treatments, while 47% (14) were for disease-specific questionnaires. All CEGs were developed for PRO questionnaires.

Figure 1: Origin of CEGs Selected For Review

RESULTS (continued)

Table 2: Examples of COA items and concept elaborations

Item Concept Elaboration

Over the past week, how embarrassed or self-conscious have you felt due to [illness]?

Recall PeriodThe phrase “over the past week” is a vital cue to patients regarding the recall period. Please translate carefully and consistently across this instrument. Please do not translate as “over the past 7 days” unless linguistically necessary. In some languages, “last week” can be confused as the previous calendar week and should be avoided.

Embarrassed: feeling ashamed or overly self-conscious. Feeling foolish or abashed.

Self-conscious: feeling overly aware of oneself, here due to the appearance of one’s skin because of illness.

Very much

A lot

A little

Not at all

These are the response options for all of the following questions. Please translate the four options such that they are all distinctly different in degree.

Very much: the respondent has experienced the problem/symptom to an extreme degree.

A lot: the respondent has experienced the problem/symptom to a great degree (but less than “very much”).

A little: not much; the respondent has experienced the problem/symptom to a small degree.

Not at all: the respondent has not experienced the problem/symptom whatsoever.

Did you experience any of the following last night?

Wheezing

Recall periodThe phrase “last night” refers to the previous night (i.e., the night prior to the day on which this instrument is administered).

Wheezing: high-pitched, whistling breaths resulting from a narrowing of the airways.

Crutches

Crutches: a staff or support used by the physically injured or disabled as an aid in walking, usually designed to fit under the arm and often used in pairs.

Built up utensils

Utensil: this is a general word for any tools used to eat food, such as a fork, spoon, chopsticks, or a table knife.

Built up: enhanced or modified for easier use or gripping. Made larger or thicker by adding materials such as tape or cloth.

Note: “built up” is preferred, if possible. If not, then “modified” is acceptable.

Does your child stack small blocks or toys on top of each other? (Spools of thread, small boxes, or toys can also be used.)

Small blocks: cubes that fit in the hand, and are easily stacked because they’re a regular shape.

Spools of thread: thread, and the bobbins/spools that the thread is wound around; picture below shows how these are good for this stacking exercise.

My chest trouble makes it hard to do certain activities, such as walking up hills, carrying things up stairs, light yardwork (such as weeding), dancing, playing bowls or golf.

Play bowls: similar to bocce ball. A light activity and lawn game in which one throws a series of medium weight balls. You may adapt this game to a sport of equivalent exertion.

Golf: You may also properly adapt “golf” to an activity of equal exertion ONLY if “golf” is not applicable in your country. Please keep in mind that golf includes lots of walking, no running, and twisting of hips.

Weeding: to pull weeds and unwanted plants in one’s garden.

DISCUSSION

Ideally, developers should provide a CEG to the LSP. When available, this developer guidance can result in fewer linguist queries during translation and harmonization and subsequently fewer revisions as a result of cognitive debriefing interviews, thereby reducing the turnaround time for linguistically validated translations. Guidance originating from developers is most ideal because it provides real-world insight from groups such as focus group and pilot study respondents, which is not available to the LSP. Guides provided by questionnaire developers analyzed in this review included the most insightful details about the intent of items, whose development was often informed by actual trial subject data. If no developer-provided CEG is available, it is ideal to have the developer review a guide created by the LSP, and provide needed modifications prior to translation.

A robust and detailed CEG should be informative for all contributors to the translation, and include item-level elaborations on each question’s intent, an explanation of recall periods, and elaboration on response sets. Additional critical elements include term-level explanations of colloquial language and guidance on conceptually equivalent alternatives for localization (e.g., of activities, household objects, garments, etc.).

CONCLUSIONS

A useful CEG includes item-level elaborations, explanations of colloquial language and guidance on localization (i.e., conceptually equivalent alternatives), as well as guidance on creating balanced response sets and clear recall periods. These elements provide valuable guidance on what each question aims to measure, which is essential for accurate data pooling across translated questionnaires. Our analysis indicates that, ideally, developers should provide these guides to the LSP, when possible. Increased developer involvement in this way, drawing on their insight from actual trial data or focus groups during questionnaire development, results in valuable guidance for linguists regarding the intent of the questionnaire items and the specific constructs they aim to measure.

REFERENCES

1. Acquadro C, Conway K, Hareendran A, Aaronson N. Literature Review of Methods to Translate Health-Related Quality of Life Questionnaires for Use in Multinational Clinical Trials. Value in Health 11.3 (2008): 509-21.

2. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, Erikson P. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: Report of the ISPOR Task Force for Translation and Cultural Adaptation. Value in Health 8 (2005): 94-104.

57% RWS created (no developer input)

23% Developer created

20% RWS created (developer reviewed)

Our review identified essential elements of CEGs to be item-level elaborations on each question’s intent, explanations of recall periods and response sets, term-level explanations of colloquial language, guidance on conceptually equivalent alternatives for localization (e.g. activities, househould objects, garments), acronym expansions, descriptions of disease and symptoms, and definitions of medical and technical terminology. Where necessary, CEGs should specify any terminology to avoid because it is not conceptually equivalent to the source concept.

Explanations of colloquial language are especially important for linguists when questionnaire items include idiomatic phrases (e.g. “Feeling blue” or “I felt like I had butterflies in my stomach”). In these situations, providing both an explanation of the term or phrase and conceptually equivalent alternatives helps to avoid a literal translation that fails to capture the meaning and nuance of the source text. COAs also often include activities, household objects, and garments, which may require adaptation prior to foreign language translation. The CEG can help to guide those modifications and substitutions.

In the case of recreational or physical activities, the CEG should provide alternative activities which are similar in movement and the level of exertion required. Similarly, disease names, symptoms, and medical and technical terminology may require localization, all benefiting from guidance prior to translation. Table 1 provides a list of essential elements for elaboration in CEGs. Table 2 lists examples of COA items and their concept elaborations.

Table 1: Essential elements for elaboration in CEGs

Symptom descriptions

Acronym expansions

Conceptually equivalent alternative terms

Guidance on unacceptable rewordings

Recall period

Medical and technical terminology definitions

Intent

Response sets

Colloquial language

Disease description

Presented at the International Society for Quality of Life Research, 25TH Annual Conference, 24 - 27 October, 2018; Dublin, Ireland.