Reliability of environmental and occupational exposure data provided by surrogate respondents in a case-control study of Parkinson's disease

Pergamon

J Clin Epidemiol Vol. 47, No. 7. 797-807. 1994 pp. Copyright 0 1994 Elsevier Science Ltd

0895_4356(94)EOO19-L Printed in Great Britain. All rights reserved 0895-4356/94 $7.00 + 0.00

RELIABILITY OF ENVIRONMENTAL AND OCCUPATIONAL EXPOSURE DATA PROVIDED BY

SURROGATE RESPONDENTS IN A CASE-CONTROL STUDY OF PARKINSON’S DISEASE

Fu-LIN WANG,’ KAREN M. SEMCHUK’ and EDGAR J. LOV$ ‘Department of Community Health Sciences Faculty of Medicine, The University of Calgary 2Centre for Agricultural Medicine, College of Medicine and College of Nursing, University of Saskatchewan and ‘Department of Community Health Sciences, Faculty of Medicine, The

University of Calgary, Calgary, Alberta, Canada T2N 4Nl

(Received in revised form 26 July 1993)

Abstract-This study used data provided by 40 non-demented Parkinson’s disease patients and 101 community controls, and by their 110 spouses and 31 adult children to assess the reliability of surrogate-provided rural environmental and occupational exposure information on the index subjects. The level of overall raw agreement between the index subjects and the spouse or adult child surrogates varied from 50.0 to 100.0% for the case-surrogate group and from 80.6 to 96.0% for the control-surrogate group. We did not detect significant differences in overall raw agreement between the case-surrogate and control-surrogate groups or between the spouse-surrogate and adult child-surrogate groups, for any of the variables studied. Considering all index subjects and their surrogates, the level of overall raw agreement was 80.3% for well water consumption, 82.3% for farm living, 85.8% for agricultural work, 87.1% for use of pesticides, 87.9% for field crop farming and 91.9% for use of fertilizers. However, the kappa estimates were lower, varying from 0.48 (SE = 0.20) for fertilizer use to 0.66 (SE = 0.11) for crop farming. The level of specific agreement was 52.2% for fertilizer use, 64.0% for pesticide use, 71.4% for agricultural work, 73.9% for crop farming, 80.9% for farm living, and 83.6% for well water consumption. The overall findings of this study support the use, if necessary, of spouses and adult children of index subjects as surrogate respondents in case-control studies of rural environmental and occupational exposures and Parkinson’s disease and, possibly, other neurologic diseases. Specific agreement seems to be a better index of reliability than overall agreement in studies where exposure is rare.

Agreement Kappa Proxy/surrogate Reliability Occupation Parkinson’s disease Case-control study

Environment

INTRODUCTION or Parkinson’s disease are among such examples. However, little is known about the

Proxy respondents are frequently used in reliability of the exposure-data provided by case-control studies as substitutes or surrogates surrogates of these patients. to provide information on exposure status when Various approaches have been used to assess the index subject is deceased, demented, or the reliability of data provided by surrogates. otherwise unable to respond. Studies of chronic The strongest study design, recommended by neurologic diseases such as Alzheimer’s disease Nelson et al. [l], uses living cases and their

791

798 Fu-LIN WANG et al.

surrogates as well as all of the controls and their surrogates. The overall percent agreement [2-lo] and the estimated kappa [3,4,7,8, 11, 121 have been frequently used as the indices of the reliability to assess the degree of deviation of the surrogate-provided data compared to the data provided by the index subjects.

Previous studies have attempted to assess, in some way, the reliability of environmental and occupational data obtained from spouse surrogates or mixed groups of surrogates [2-131. These studies examined the overall percent agreement and estimated kappa values between mixed groups of index subjects (i.e. including both cases and controls) and their surrogates [5-8, 10, 111 and the reliability of data provided by surrogates of cases [2,3,9, 131 or controls [ 121. Three studies related to Alzheimer’s disease [2,4,12] have examined, to some extent, the reliability of surrogate-provided environmental exposure data (drinking water and living near a waste dump [4], alcohol consumption and smoking [2], diet [4, 121 and animal contact [2,4, 121, occupational exposure data (exposure to pesticides and paints/varnish [4]) and occupational histories (number of jobs [2,4], last job [2], job stress and protection [4]). A review of all published epidemiologic and etiologic studies of Parkinson’s disease indexed in Index Medicus, to date, yielded no previous studies that have attempted to assess the reliability of environmental or occupational exposure data provided by surrogates of patients with Parkinson’s disease.

This study used data provided by patients with Parkinson’s disease and community controls and by spouses and adult children of these index subjects, to assess the reliability of the surrogate-provided rural environmental and occupational exposure data on the index subjects. Three measurements were used: the overall percent agreement, the kappa statistic and the specific percent agreement [14]. The last index, which is defined as the percent agreement between the index subjects and their surrogates on a positive history of an exposure, is not frequently seen in literature although it may be useful in the assessment of the reliability of surrogate-provided data in studies where exposure is rare. We compared the differences in the percent agreement and the estimated kappa values between the spouse surrogates and the adult child surrogates within two groups (the case-surrogate group and the control-surrogate group), as well as the differences between the

case-surrogate and control-surrogate groups. This paper extends our analysis, reported earlier [ 151, in which we assessed the usefulness of the demographic data provided by the same surrogates.

MATERIALS AND METHODS

Study group and data collection

The exposure data on the cases and controls were collected as part of a larger study of Parkinson’s disease [16-l 81. The cases were selected from a population-based case register of Calgary residents with neurologist-confirmed idiopathic Parkinson’s disease. Only living and non-demented cases with idiopathic Parkinson’s disease were recruited into the study. The controls were selected from the Calgary general population of random-digit-dialling [ 191. The methods for the selection of the cases, the controls and their surrogates, and for data collection have been described, in detail, else- where [15-181. In brief, each case and control was asked to identify a surrogate, preferably a spouse, or the eldest child, or a sibling who could be interviewed as a proxy respondent for the index subject. The cases and controls and their surrogates were personally interviewed in exactly the same manner by trained experienced interviewers, using a structured questionnaire. Both the interviewers and the respondents were blind as to the underlying hypotheses of the study. Three categories of information were investigated: (1) demographic data (including date of birth, sex, marital status, educational level, annual family income, ethnic background, and household size); (2) environmental exposure information, such as the location (city or town, farm or acreage) and dates of each residence, and the source of the drinking water; and (3) lifetime occupational history (i.e. exposure dates and descriptive information on all work-related contacts with pesticide and fertilizer chemicals). The data were reviewed and coded by one of the authors (KMS).

Data were collected on 43 case-surrogate pairs and on 109 control-surrogate pairs. The study sample represents 33.1% of the total sample of 130 case-surrogate pairs and 30.1% of the total sample of 362 control-surrogate pairs (the 260 controls included in the Parkin- son’s case-control study plus 102 controls orig- inally matched with Parkinson’s patients who were later excluded from the casecontrol

Reliability of Surrogate-provided Data 799

study). Because of the small number of siblings interviewed, the analysis was restricted to the I 10 spouse and 3 1 adult child surrogate-index subject pairs. Of these, 36 spouses and 4 adult children comprised the case-surrogate pair group, and 74 spouses and 27 adult children comprised the control-surrogate pair group.

Data analysis

The environmental exposure variables included both farm living and well water drinking during the first 45 years of life, and during five other exposure periods (age <5, 6-15, 16-25, 26-35 and 3645) of interest. The occupational exposure variables included exposure between age 16 and 55 to agricultural work, field crop farming, synthetic fertilizers (either in solid or liquid form), herbicides, insecticides, fungicides, and to pesticides (i.e. herbicides, insecticides or fungicides), during four exposure periods of interest: age 16-25, 26-35, 3645 and 46-55 years. Each exposure period of interest was represented by a separate binary variable, coded as “exposed” or “not exposed”.

The analysis focused on the assessment of misclassification bias, using the overall percent agreement, the specific percent agreement and the kappa statistic. All of the calculations for these three indices were based on Fleiss’s for- mula [14]. Criteria for judging the level of percent agreement or concordance, as used by others [2,4], are greater than 80% for excellent, 61-80% for good, 41-60% for moderate and less than 41% for poor. To interpret the clinical significance of the estimated kappa values, we used the “benchmarks” (shown below) suggested by Landis and Koch [20] as a “consistent nomenclature when describing the relative strength of agreement associated with kappa statistics”.

Kappa Statistics Strength of Agreement <o.oo Poor 0.00~.20 Slight 0.21-0.40 Fair 0.414.60 Moderate 0.61-0.80 Substantial 0.81-1.00 Almost perfect

We compared the differences in the level of overall agreement and specific agreement between the case-surrogate group and the control-surrogate group and between the index subject-spouse surrogate group and the index subject-child surrogate group. The cases and controls were not matched in this analysis. The index subject-surrogate group, where the

surrogate was the spouse, and the index subject-surrogate group, where the surrogate was the eldest child, were also independent groups. Hence, we used Fisher’s exact test (two-sided), where appropriate, and the unmatched chi- square test for these analyses to make statistical inference. The estimated kappa values are pre- sented with their standard error (SE). Statistical power was calculated by the methods outlined by Fleiss [14] and is reported with an a = 0.05. All analyses were done using the SPSS” (version 2.2) and Minitab (version 7.2) software pack- ages on IBM-compatible (Zenith) and main- frame (NOS/VE) computers.

RESULTS

Response pat tern

Table 1 shows the response rates for the nine variables examined by study group. All of the cases and controls provided information on exposure to farm living, well water consumption, agricultural work, field crop farming and pesticide use. The response rate for the other variables varied, for the cases from 95.3% for fertilizer use to 100.0% for fungicide use and for the controls from 97.3% for herbicide use and insecticide use to 99.1% for fungicide use. The response rates were nearly as high for the surrogates, varying in the case-surrogate group from 93.0% for fertilizer use to 100.0% on five of the nine variables examined, and in the control-surrogate group from 95.4% for herbicide use to 100.0% for five of the nine variables studied. There are no significant differences in the response rates for any of the variables studied between the cases and the controls, between the cases and their surrogates, between the controls and their surrogates, or between the case-surrogates and the control-surrogates.

As shown in Table 2, the reported rates of lifetime environmental exposure to farm living and well water consumption varied from 42.5 to 52.5% in the cases and from 50.5 to 69.1% in the controls. The exposure rates for all seven occupational exposures are 30% or less in the cases and below 23.0% in the controls.

The overall percent agreement and kappa statistic for agreement between the index subjects and their surrogates on the environmental and occupational exposures of the index subjects

Table 3 summarizes the overall percent agreement and the estimated kappa values for all


Table 1. Response rates* of all study subjects by exposure variable and study group

Study group

Case- Control- Cases surrogates Controls surrogates

Variable n % n % n % n %

(A ) Environmental Exposures Farm living Well water consumption

(B) Occupational Exposures Agricultural work Field crop farming Fertilizer use Pesticide use Herbicide use Insecticide use

43 100.0 43 100.0 109 100.0 109 100.0 43 100.0 43 100.0 109 100.0 105 96.3

43 100.0 43 100.0 109 100.0 109 100.0 43 100.0 43 100.0 109 100.0 I09 100.0 41 95.3 40 93.0 107 98.2 106 97.3 43 100.0 42 97.7 I09 100.0 I09 100.0 42 97.7 41 95.3 106 97.3 104 95.4 42 97.7 41 95.3 106 97.3 108 99. I

Fungicide use 43 100.0 43 100.0 108 99. I 109 100.0

*All chi-square tests for differences in response rates between the cases and the controls (x f,,), between the cases and their surrogates (x f,,), between the controls and their surrogates (x t,,, between the case-surrogates and the control-surrogates (x f,,), and across all four response groups (x b,), are not statistically significant, i.e. p > 0.05.

lifetime environmental and occupational exposures, by type of surrogate in the case-surrogate and control-surrogate groups. The level of overall agreement on the two environmental exposures between the index subjects and their spouse surrogates or adult child surrogates varies from 50.0 to 77.8% for the case-surrogate group and from 80.6 to 88.9% for the control-surrogate group. The estimated kappa values vary from 0.20 to 0.54 in the case-surrogate group and from 0.61 to 0.78 in the control-surrogate group.

Compared to the environmental variables, the levels of overall agreement on the seven occupational exposures are higher; varying from 75.0 to 100.0% for the case--surrogate group and from 85.2 to 96.0% for the control-surrogate group. The estimated kappa values vary from 0

Table 2. Lifetime exposure rate for cases and controls by exposure variable

Cases Controls

Variable niN % l nlN % *

(A) Environmental Exposures Farm living l7/40 42.5 5ljlOl 50.5 Well water consumption 21/40 52.5 67/97 69. I

(B) Occupational Exposures Agricultural work l2/40 30.0 23/101 22.8 Field crop farming II/40 27.5 21/101 20.8 Fertilizer use 8137 21.6 5/98 5.1 Pesticide use IO/39 25.6 ISjlOl 14.9 Herbicide use 7/38 18.4 7194 7.5 Insecticide use 5/38 13.2 IO/98 10.2 Fungicide use 4/40 10.0 7/lOl 6.9

*The percent of cases or controls who reported exposure to the environmental factors during any period of the first 45 years or to the occupational factors during any period between the ages of 16-55 years.

to 0.71 for the case-surrogate group and from -0.04 to 0.78 for the control-surrogate group. Of note, some of the estimated kappa values are not significantly different from zero, including the kappa estimates on (1) all variables in the case-child surrogate group and the seven occupational variables in the control-child surrogate group, (2) fungicide use in the case-spouse surrogate group and the four occupational variables (fertilizer use, herbicide use, insecticide use and fungicide use) in the control-spouse surrogate group, and (3) fungicide use in the case-all surrogate group and fertilizer use or insecticide use in the control-all surrogate group.

We did not detect a statistically significant difference in the level of overall agreement, for any of the nine variables studied, between the case-surrogate and the control-surrogate groups when the spouse surrogate group and the child surrogate group were considered sep- arately or together. Similarly, for both the case-surrogate group and the control-surrogate group, we did not detect a statistically significant difference in the level of overall agreement between the spouse surrogates and the adult child surrogates for any of the variables studied. However, the statistical power of these tests was low. For the spouse surrogate and adult child surrogate comparisons, the power was between 3.7 and 7.3% for the tests of differences (3.8-27.8%) for the two environmental variables and from 0 to 17.2% for the tests of differences (l.l-16.7%) for ,the occupational variables. For the case-surrogate group vs control-surrogate group comparisons, the observed differences in the level of overall agreement

between the groups while the statistical from 0.1 to 62.1%.


varied from 0.6 to 38.9%, Considering all index subjects (combining the power of the tests varied cases and the controls) and their surrogates,

occupational exposure to fertilizers resulted in

Table 3. The overall percent agreement and kappa estimate on lifetime environmental and occupational exposures between index subjects (cases/controls) and their surrogates by the relationship of the surrogate to the index

subject

Relationship of surrogate to index subject

Case-surrogate pairs Control-surrogate pairs

Agreement Kappa Agreement Kappa N (%) est. SET N (%) est. SEt FET*

(A) Environmental exposures 1. Farm living

Spouse 36 15.0 0.498 Child 4 50.0 0.200 All 40 72.5 0.450 FET$ p = 0.754

2. Well water consumption Spouse 36 77.8 0.544 Child 4 50.0 0.200 All 40 75.0 0.494 FETJ p = 0.999

(B) Occupational exposures 3. Agricultural work

Spouse 36 Child 4 All 40 FETf

4. Field crop farming Spouse 36 Child 4 All 40 FETf

5. Use of fertilizers Spouse 34 Child 3 All 37 FETS

6. Use of pesticides Spouse 35 Child 4 All 39 FETS

6.1 Use of herbicides Spouse 34 Child 4 All 38 FET$

6.2 Use of insecticides Spouse 34 Child 4 All 38 FET$

6.3 Use of fungicides Spouse 36 Child 4 All 40 FET$

83.3 0.640 100.0 N/A§ 85.0 0.659

p = 0.607

86.1 0.696 100.0 N/A5 87.5 0.711

p = 0.999

85.3 0.608 100.0 N/A5 86.5 0.619

p = 0.999

85.7 0.679 75.0 0.000 84.6 0.643

p = 0.999

88.2 0.640 75.0 0.000 86.8 0.585

p = 0.999

91.2 0.617 100.0 N/AI 91.9 0.621

p = 0.999

83.3 0.413 100.0 N/A§ 84.6 0.423

p = 0.999

0.168 74 0.387 27 0.158 101

0.171 72 0.387 25 0.160 97

0.180 74 - 27

0.178 101

0.182 74 - 27

0.181 101

0.221 73 - 25

0.222 98

0.189 74 0.866 27 0.184 101

0.246 70 0.866 24 0.238 94

0.313 71 - 27

0.315 98

0.265 74 - 27

0.266 101

85.1 88.9 86. I

0.704 0.116 p =0.291 0.779 0.193 p =0.112 0.723 0.099 p =0.085

p = 0.560

80.6 88.0 82.5

0.607 0.119 p =0.999 0.737 0.219 p = 0.349 0.640 0.104 p = 0.779

p = 0.507

86.5 0.618 85.2 0.514 86.1 0.594

p = 0.999

87.8 0.633 88.9 0.601 88. I 0.626

p = 0.743

94.5 0.000 92.0 -0.042 93.9 -0.017

p = 0.999

86.5 0.426 92.6 0.625 88.1 0.472

p = 0.507

94.2 0.471 96.0 0.779 94.7 0.587

p = 0.999

90.1 0.315 88.9 0.509 89.8 0.389

p = 0.999

94.6 0.571 91.3 0.452 93.8 0.538

p = 0.625

0.157 p =0.774 0.291 p = 0.628 0.138 p = 0.999

0.165 p = 0.999 0.310 p = 0.628 0.146 p =0.999

0.486 p = 0.139 0.693 p = 0.999 0.399 p = 0.288

0.210 p = 0.999 0.388 p = 0.349 0.185 p =0.719

0.343 p = 0.434 0.426 p = 0.261 0.268 p = 0.999

0.290 p = 0.999 0.356 p = 0.999 0.226 p = 0.999

0.306 p = 0.077 0.480 p = 0.999 0.258 p = 0.999

*Fisher’s exact test (two-sided) for the difference in the overall agreement between the case-surroaate nairs and the control-surrogate pairs by the surrogate’s relationship to the index subject.

_ .

TStandard error of the estimated kappa value. JFisher’s exact Test (two-sided) for the difference in overall agreement between the spouse surrogate and adult

child surrogate groups within the case/control-surrogate pair group. $Unable to calculate.


the highest level of overall agreement (91.9%), followed by crop farming (87.9%), pesticide use (87.1%), agricultural work (85.8%), farm living (82.3%) and well water consumption (80.3%). All of these would be considered as excellent agreement [2,4]. In contrast, the estimated kappa values associated with these lifetime exposure variables were lower and using Landis and Koch’s criteria [20] would have been considered as substantial for crop farming (0.66 + 0.1 l), farm living (0.64 f 0.09) and agricultural work (0.62 + 0.1 1), and as moderate for well water consumption (0.59 f 0.09), pesticide use (0.56 f 0.13) and fertilizer use (0.48 + 0.20).

Spec@c agreement between the index subjects and their surrogates on the environmental and occupational exposures of the index subjects

As shown in Table 2, all of the lifetime occupational exposure rates were less than 30%. Because of this, we investigated the specific agreement (Table 4). For the case-spouse surrogate group, the level of specific agreement on the nine factors varied from 50.0 to 81 .O%. In contrast, probably because of the small sample size, the levels of specific agreement between the cases and the adult child surrogates are 50.0% for the two environmental exposures and 0% for all seven occupational exposure variables. Although there were large differences (22.7-80.0%) in the level of specific agreement between the spouse surrogate and adult child surrogate groups in the case group, the differences were not statistically significant. The statistical power of these tests ranged from 8.1 to 98.7%.

Within the control-surrogate group, the differences between the spouse surrogate and adult child surrogate groups varied from 0% for the variable fertilizer use to 30.0% for the variable herbicide exposure. None of these differences were statistically significant. How- ever, the statistical power of the study to detect the observed differences was 65.9% or less. Comparing the case and control-spouse surrogate groups, we found a significantly higher level of specific agreement within the case group for the variable exposure to fertilizers. We did not detect significant differences in the level of specific agreement between the case and control surrogate groups for any of the other variables. However, the statistical power for these tests was less than 80%. Comparing the case and the control-adult child surrogate groups, there were no significant differences in the level of specific

agreement between the groups for any of the variables studied. The statistical power for all of the tests conducted was less than 50% except for the variable herbicide use (97.0%).

When we combined the spouse surrogate and adult child surrogate groups, with the exception of fertilizer use, the levels of specific agreement did not differ significantly between the case-surrogate and control-surrogate groups. However, the statistical power was 4.6% for detecting the lowest observed difference (5.2%) for herbicide use and 57.2% for detecting the highest observed difference (22.3%) for insecticide use.

Comparing all index subjects and their surrogates, the level of specific agreement was 52.2% for fertilizer use, 64.0% for use of pesticides, 71.4% for agricultural work, 73.9% for crop farming, 80.9% for farm living and 83.6% for well water consumption.

The reliability of surrogate-provided exposure data by exposure period

Table 5 shows the level of overall agreement, the estimated kappa values for the agreement and the level of specific agreement between the index subjects and their surrogates for the case-surrogate and control-surrogate groups, by exposure factor and exposure period (age of the index subject). In this analysis, the spouse surrogates and adult child surrogates are combined. For most of the exposure variables, we observed no consistent trend in the level of agreement, as estimated by the overall agreement, the estimated kappa values and the specific agreement between the case-surrogate and control-surrogate groups across the exposure periods of interest. However, within the control-surrogate group the level of overall percent agreement between the control subjects and their surrogates on past exposure to agricultural work, field crop farming, fertilizers and pesticides appears to improve as the exposure periods of interest become more recent. The same trend appears within the case-surrogate group for past exposure to field crop farming.

We did not detect a significant difference in the level of overall agreement between the case-surrogate and control-surrogate groups for any exposure period, for any of the variables studied. However, the statistical power of the tests to detect the observed differences (which varied from 0.2 to 11.4%) was 37.2% or less. Similarly, we did not detect a significant difference in the level of specific agreement between


Table 4. The specific percent agreement* on lifetime exposures? between index subjects (cases/controls) and their surrogates by the relationship of surrogate to the index subject and by exposure variable

Relationship of surrogate Case-surrogate pairs Control-surrogate pairs to index FETt subject ++§ + - II - +** --tt (%) ++§ +-II - +** --tt (%I (A) Environmental exposures 1. Farm living

Spouse 12 4 Child 1 0 All I3 4 FETq

2. Well water consumption Spouse 17 3 Child 1 0 All 18 3 FETY

(B) Occupational exposures 3. Agricultural work

Spouse IO 2 Child 0 0 All 10 2 FET?

4. Field crop farming Spouse 10 1 Child 0 0 All 10 I FET?l

5. Use of fertilizers Spouse 6 2 Child 0 0 All 6 2 FETq

6. Use of pesticides Spouse 9 I Child 0 0 All 9 I FETB

6.1 Use of herbicide Spouse 5 2 Child 0 0 All 5 2 FETg

6.2 Use of insecticide Spouse 3 2 Child 0 0 All 3 2 FETI[

6.3 Use of fungicide Spouse 3 I Child 0 0 All 3 1 FETT[

5 2 7

p = 0.567

5 2 7

p = 0.202

4 0 4

N/A

4 0

N;A

3 0

N; A

4 I 5

p = 0.250

2 1 3

p = 0.333

I 0 I

N/A

5 0

N; A

15 72.1 29 9 I 50.0 11 2

16 70.3 40 II

11 81.0 36 13 I 50.0 15 3

12 78.3 51 16

20 76.9 4 0.0

24 76.9

I2 3

I5

II 3

14

0 0 0

5 2 7

2 2 4

2 2 4

3 1 4

5 3 8

21 80.0 4 0.0

25 80.0

5 2 7

23 70.6 3 0.0

26 70.6

4 1 5

21 78.3 3 0.0

24 75.0

7 1 8

25 71.4 3 0.0

28 66.7

3 0 3

28 66.7 4 0.0

32 66.7

5 I 6

27 50.0 3 0.0

30 50.0

2 I 3

2 1 3

p = 0.753

1 0 I

p = 0.393

5 I 6

p = 0.701

4 1 5

p = 0.999

0 1

N; A

3 I 4

p = 0.652

I I 2

p = 0.565

2 2 4

p = 0.631

2 I 3

p = 0.999

34 84.1 p =0.192 13 88.0 p =0.127 47 85.1 p = 0.082

22 83.7 p =0.804 7 90.9 p = 0.080

29 85.7 p = 0.348

52 70.6 p = 0.769 20 60.0 N/A 72 68.2 p = 0.586

54 71.0 p = 0.541 21 66.7 N/A 75 70.0 p = 0.404

69 0.0 p = 0.021 23 0.0 N/A 92 0.0 p = 0.005

59 50.0 p = 0.064 23 66.7 p = 0.429 82 53.9 p = 0.149

63 50.0 p = 0.387 22 80.0 p = 0.333 85 61.5 p = 0.999

62 36.4 p = 0.370 22 57.1 N/A 84 44.4 p = 0.420

67 60.0 p = 0.691 20 50.0 N/A 87 57.1 p = 0.999

*Percent agreement on a positive history of exposure between the index subject and the surrogate. TEnvironmental exposures during any period of the first 45 years of life, occupational exposures at any period between

the ages of 16-55 years. SFisher’s exact test (two-sided) for the difference in specific agreement between the case-surrogate pairs and the

control-surrogate pairs. $An index subject states exposed, a surrogate states exposed. /IAn index subject states exposed, a surrogate states not exposed. **An index subject states not exposed, a surrogate states exposed. ttAn index subject states not exposed, a surrogate states not exposed. TFisher’s exact test (two-sided) for the difference in specific agreement between the spouse surrogate group and the adult

child surrogate group within the case/control-surrogate pair group. N/A: Unable to calculate.


Table 5. Agreement and kappa estimate between index subjects (cases/controls) and their surrogates by exposure variable and exposure period

Exposure variable & period

Case-surrogate group Control-surrogate group

N OPAt Kappa SEt SPAIj N OPAt Karma SEi SPAG FET*

1. Farm living Age

<5 40 80.0 6-15 38 84.2 16-25 38 84.2 26-35 39 79.5 36-45 39 87.2

2. Well water consumption Age

c5 24 95.8 6-15 24 87.5 16-25 31 77.4 2635 34 76.5 36-45 38 84.2

3. Agricultural work Age _

1625 39 87.2 26-35 40 90.0 3645 40 97.4 46-55 40 95.0

4. Field crop farming Age

1625 39 87.2 2635 40 36-45 40 4655 40

5. Use of fertilizers Age

16-25 38 26-35 38 3645 38 46-55 37

6. Use of pesticides Age

I625 38 2635 39 36-45 39 4655 39

90.0 95.0 95.0

89.5 0.539 0.298 60.0 99 96.0 0.315 0.401 33.3 p = 0.217 94.7 0.638 0.393 66.7 99 98.0 0.490 0.495 50.0 p = 0.576 94.7 0.638 0.393 66.7 99 99.0 0.0 0.995 0.0 p =0.186 91.9 0.373 0.427 40.0 98 99.2 N/All N/All 0.0 p = 0.063

84.2 0.409 0.269 50.0 100 86.0 84.6 0.478 0.248 57.1 100 92.0 84.6 0.316 0.297 40.0 99 92.9 94.9 0.480 0.484 50.0 99 93.9

0.597 0.159 77.8 93 81.7 0.626 0.106 78.5 p = 0.999 0.671 0.169 80.0 93 86.0 0.704 0.110 81.7 p = 0.999 0.662 0.173 78.6 94 84.0 0.670 0.107 80.0 p = 0.999 0.520 0.185 66.7 98 86.7 0.664 0.125 75.5 p = 0.302 0.588 0.238 66.7 98 90.8 0.655 0.168 71.0 p = 0.540

0.917 0.204 95.7 66 86.4 0.726 0.124 87.3 p = 0.279 0.750 0.204 87.0 68 83.8 0.678 0.121 84.1 p = 0.755 0.545 0.181 80.0 74 83.8 0.674 0.117 85.7 p = 0.579 0.436 0.203 60.0 86 84.9 0.638 0.127 74.5 p = 0.294 0.406 0.270 50.0 91 94.4 0.793 0.202 81.8 p = 0.083

0.677 0.198 76.2 99 84.9 0.554 0.140 65.1 p = 0.796 0.543 0.299 60.0 100 96.0 0.729 0.241 75.0 p = 0.225 0.655 0.566 66.7 IO1 98.0 0.490 0.495 50.0 p = 0.999 0.474 0.488 50.0 IO1 100.0 N/All N/All 0.0 p = 0.079

0.677 0.198 76.2 100 86.7 0.599 0.144 68.3 p =0.999 0.444 0.338 50.0 100 96.0 0.646 0.280 66.7 p = 0.225 0.474 0.488 50.0 IO1 99.0 0.662 0.573 66.7 p = 0.194 0.474 0.488 50.0 I01 100.0 N/All N/All 0.0 p = 0.079

0.340 0.193 41.7 p = 0.999 0.460 0.240 50.0 p = 0.216 0.186 0.326 22.2 p =0.192 0.221 0.346 25.0 D = 0.999

*Fisher’s exact test (two-sided) for the difference in overall percent agreement between the case-surrogate group and the control-surrogate group by exposure period.

tOvera percent agreement (OPA). @tandard error of the estimated kappa value. $Specific percent agreement (SPA). flUnable to calculate.

the case- and control-surrogate groups for any of the variables studied. These non-significant findings could be the result of the low statistical power, which varied from 3.8 to lOO.O%, to detect the observed differences of 0.7-25.0% in the level of specific agreement between the case- and control-surrogate groups.

DISCUSSION

This study attempted to assess the reliability of environmental and occupational exposure data provided by surrogates (110 spouses and 3 1

adult children) for index subjects (40 non-demented patients with Parkinson’s disease and 101 community controls). The overall agreement, the kappa statistic and the specific agreement were used as indices to compare the reliability of data provided by spouse surrogates and by the adult child surrogates, and to compare the reliability of exposure data provided by surrogates of cases and by surrogates of controls. To our knowledge, this is the first study to examine the reliability of exposure-data provided by surrogates of patients with Parkinson’s disease.

Reliability of Surrogate-provided Data x05

The level of overall agreement between the With respect to the reliability of the proxy combined group of index subjects and their responses for the case-surrogate group vs the surrogates on environmental and occupational control-surrogate group, we did not detect a exposures was excellent, varying from 80.3% for significant difference in the level of overall or well water consumption to 91.9% for the use of specific agreement between the two groups for fertilizers. These findings are comparable to any exposure period, for any of the variables those reported earlier for the demographic vari- examined. Our findings are consistent with ables [15]. Our findings also resemble those those reported by Herrmann et al. [8], that the reported by Rocca et al. [2] and Lerchen and correlation coefficients of responses for most of Samet [3]. Considering the possibility of chance- the variables examined were similar for agreement when percent agreement is used, we case-surrogates and control-surrogates. While also used the kappa statistic in our study. As a we cannot rule out the alternative hypothesis of result, the levels of agreement (kappa esti- low statistical power, the non-significant differ- mate f SE) between the combined group of ences in the levels of agreement between the index subjects and their surrogates are slightly case-surrogate and control-surrogate groups lower, varying from substantial for crop farm- might suggest that the quality of data provided ing (0.66 f 0.1 l), farm living (0.64 + 0.09) and by the surrogates of cases and the surrogates of agricultural work (0.62 + 0.11) to moderate controls in our study are similar. The similar for well water consumption (0.59 f 0.09), response rates of the case-surrogate and con- pesticides use (0.56 f 0.13) and fertilizers use trol-surrogate groups for all variables studied (0.48 + 0.20). further support this view.

The nature of the relationship of the proxy respondent to the index subject may affect the quality of data provided by the surrogate [ 1, 131. This study compared the reliability of rural environmental and agricultural occupational data provided by spouse surrogates and adult child surrogates. For all of the variables studied, we did not detect a statistically significant difference in the levels of overall agreement and specific agreement between the spouse surrogates and adult child surrogates in both case-surrogate and control-surrogate groups. As the observed differences in overall agreement (l. l-27.8%) and especially in specific agreement (O-80%) between the comparison groups are large, we cannot rule out inadequate statistical power as a possible explanation for these non-significant findings. The statistical power of the study to detect the observed differences between the groups did not surpass 17.2% for the comparisons of overall agreement and varied from 0 to 98.1% for the tests on specific agreement. Comparison of the estimated kappa values and corresponding standard errors between the spouse and adult child surrogates resulted in differences of 0.03-0.68, most of which are not significant (p > 0.05). Notwith- standing the important issue of statistical power, it is interesting that in their National Health and Nutrition Survey, McLaughlin et al. [lo], also found that spouses and children were equally reliable informants for the two smoking- related variables, smoking status and age started smoking.

Although the level of overall agreement between the cases or controls and their surrogates (spouse and adult child surrogates combined) in our study exceeded 80% for most of the variables studied, the level of specific agreement and the estimated kappa values were lower. For instance, the majority of kappa statistics for agreement between the cases or the controls and their surrogates indicate a moderate level of agreement or lower; none of the estimated kappa values indicate an almost perfect level of agreement. This discrepancy, which is particu- larly large for the seven occupational exposures of interest, may be due to the low frequencies of some exposures [21]. In this study, the reported prevalence of the seven lifetime occupational exposures amongst the cases varied from 10.0% for fungicide use to 30.0% for agricultural work; amongst the controls the rates were lower, varying from 5.1% for fertilizer use to 22.8% for agricultural work. The corresponding levels of specific agreement and kappa estimates for the two lowest-prevalence exposure variables were also low, at 50.0% and 0.42, respect- ively, for fungicide use in the case-surrogate group and 0.0% and -0.02 for the variable fertilizer use in the control-surrogate group. As indicated in Table 4, because there is greater agreement on the absence of exposure (cell d) for each of the seven occupational exposure variables, the level of overall agreement is likely to be inflated. For example, although the use of fertilizers resulted in the highest level of overall agreement (91.9%) between all index subjects


and their surrogates, the variable also resulted in the lowest level of specific agreement (52.2%) and the smallest kappa estimate (0.48). This observation may have implications for data obtained from surrogates in studies where exposure is rare. In such studies, specific agreement may be a better index of the reliability of surrogate-provided data. Thus, surrogate-provided data with a high level of specific agreement may have less of an impact, if any, on the odds ratio than would data with the same level of overall agreement.

It should be borne in mind that the measures of reliability of surrogate-provided data are not “perfect”. While the overall percent agreement is the simplest and most commonly used measure, the index does not account for chance. In addition, as illustrated by Rogot and Gold- berg [22], the same value of the overall agreement does not necessarily reflect an equally good agreement. Compared with the overall percent agreement, the specific agreement on the presence of an exposure (cell a) is not inflated when exposure is rare. Also, the level of specific agreement often indicates the same level of agreement as the kappa statistic. Like overall agreement, however, specific agreement may also include a degree of chance-agreement. The kappa statistic has been developed to take chance-agreement into consideration and is widely used. However, the high sensitivity of the kappa statistic to the prevalence of exposure often prevents the straightforward comparison of kappa estimates across studies in which the prevalence of exposure might differ [l]. The kappa estimates are unstable and tend to be lower when the prevalence of exposure is ex- tremely low or high [21,23]. When the sample size is small, this problem becomes worse. In our study, we were unable to estimate five kappa values for the adult child surrogates in the case-surrogate group and over half of the kappa values for the control-surrogate group on the occupational exposures were not significant. Thus, we found it informative to use more than one measure of agreement to assess the reliability of surrogate-provided data in our study. A more detailed discussion on the measurement of the quality of surrogate-provided information and other issues concerning the use of surrogates in case-control studies may be found in Walter and Irwig’s paper [23] and other recent publications [l, 21,241.

Because exposure may vary by the subject’s age, risk estimation by exposure period is

necessary. In this study, for most of the exposure variables examined, we observed no consistent trend in the levels of agreement, estimated by overall agreement, kappa statistic and specific agreement across the exposure periods of interest. However, the level of overall agreement between the index subjects and their surrogates appears to improve as the exposure periods of interest become more recent for occupational exposure to field crop farming within the case-surrogate group and for four occupational exposures (agricultural work, field crop farming, fertilizer use and pesticide use) within the control-surrogate group. The marked decline in the frequency of reported occupational exposures with age, for both the cases and controls, could account for this appar- ent trend in the level of overall agreement across the exposure periods. Because each exposure period of interest was represented by a separate binary variable, we were unable to conduct a test for trend across the exposure periods.

In case-control studies of occupational or environmental factors associated with industries that are regulated by departments of Occu- pational Health and Safety, and for which the noxious agents and their exposure levels are well known and documented, when index subjects are unable to provide their own work-related exposure history, there may be better sources of information than can be provided by familial surrogates, such as work or government records, or former co-workers. In Canada, the agriculture industry is not regulated by Occu- pational Health and Safety. The majority of farms have been and continue to be family run enterprises. Records on pesticide and fertilizer usage during the exposure periods of interest for this older study group are virtually non-existent. Even today, little is known about the extent of occupational or environmental agrochemical exposures on the family farm. Industrial hygiene and toxicological studies are just beginning in this area. Hence, at present, in case-control studies of past agricultural or rural environmental exposures, for index subjects who are unable to provide their own exposure histories, our best source of exposure information is usually a close family member such as the spouse or adult child.

In this study, we examined the reliability of exposure data provided by surrogates of both cases and controls, and compared three indices of agreement on exposure status between the index subjects and their surrogates. We found


that the spouses and adult children of the index subjects in our case-control study of Parkin- son’s disease were fairly reliable informants concerning the index subjects’ past exposures to various rural environmental and agricultural occupational factors. The findings support the use, if necessary, of spouses and adult children of index subjects as surrogate respondents in case-control studies of rural environmental exposures (farm living, well water consumption) and agricultural occupational exposures (agricultural work, field crop farming, fertilizer use and pesticide use) and Parkinson’s disease and, possibly, other neurologic diseases. However, further studies are still needed to fully explicate the relationship between the magnitude of the kappa estimate and the level of percent agreement (overall or specific) on rural environmental and agricultural exposure factors.

Acknowledgemenrs-This research was supported by a grant from the Parkinson Foundation of Canada. F-L.W. was supported by Alberta Heritage Foundation for Medical Research.

1.

2.

3.

4.

5.

6.

REFERENCES

Nelson LM, Longstreth WT Jr, Koepsell TD, van Belle G. Proxy respondents in epidemiologic research. Epidemiol Rev 1990; 12: 71-86. Rocca WA, Fratiglioni L, Bracco L, Pedone D, Groppi C, Schoenberg BS. The use of surrogate respondents to obtain questionnaire data in case<ontrol studies of neurologic diseases. J Chron Dis 1986; 39: 907-912. Lerchen ML, Samet JM. An assessment of the validity of questionnaire responses provided by a surviving spouse. Am J Epidemiol 1986; 123: 481489. dhong JP, Turpie I, Haines T, Muir G, Farnworth H, Cruttenden K, Julian J. Verma D. Hillers T. Concor- dance of occupational and environmental exposure information elicited from patients with Alzheimer’s disease and surrogate respondents. Am J Ind Med 1989; 15: 73-89. Rogot E, Reid DD. The validity of data from next-of- kin in studies of mortality among migrants. Int J Epidemiol 1975; 4: 51-54. Kolonel LN, Hirohata T, Nomura AMY. Adequacy of survey data collected from substitute respondents. Am J Epidemiol 1977; 106: 476484.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

Marshall J, Priore R, Haughey B, Rzepka T, Graham S. Spouse-subject interviews and the reliability of diet studies. Am J Epidemiol 1980; 112: 675-683. Herrmann N. Retrospective information from ques- tionnaires. I. Comparability of primary respondents and their next-of-kin. Am J EDidemiol 1985: 121: 936-947. Samet JM. Surrogate measures of dietary intake. Am J CIin Nutr 1989; 50: 1139-1144. McLaughlin JK, Dietz MS, Mehl ES, Blot WJ. Re- liability of surrogate information on cigarette smoking by type of informant. Am J Epidemiol 1987; 126: 144-146. Metzner HL, Lamphiear DE, Thompson FE, Oh MS, Hawthorne VM. Comparison of surrogate and subject reports of dietary practices, smoking habits and weight among married couples in the Tecumseh diet method- ology study. J Clin Epidemiol 1989; 42: 367-375. Heyman A, Wilkinson WE, Stafford JA, Helms MJ, Sigmon AH, Weinberg T. Alzheimer’s disease: A study of epidemiological aspects. Ann Neural 1984; 15: 335-341. Pickle LW, Brown LM, Blot WJ. Information avail- able from surrogate respondents in case-control inter- view studies. Am J Epidemiol 1983; 118: 99-108. Fleiss JL. Statistical Methods for Rates and Proportions 2nd ed. New York: Wiley; 1981: 4446, 212-225. Wang FL, Semchuk KM, Love EJ. An assessment of usefulness of demographic data provided by surrogate respondents in a case-control study of Parkinson’s disease. J Clin Epidemiol 1992; 45: i219-1227. Semchuk KM, Love EJ. Lee RG. Parkinson’s Disease and exposure to rural environmental factors: A population based case-control study. Can J Neural Sci 1991; 18: 279-286. Semchuk KM, Love EJ, Lee RG. Parkinson’s disease and exposure to agriculture work and pesticide chemicals. Neurology 1992; 42: 1328-1335. Semchuk K@ Love EJ, Lee RG. Parkinson’s disease: A test of the multifactorial etiologic hypothesis. Neu- rology 1993; 43: 1173-l 180. Waksberg J. Sampling methods for random digit dialling. J Am Stat Assoc 1978; 73: 40-46. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174. Walter SD. Measuring the Reliability of Clinical Data: The Case for Using Three Observers. Rev Epidemiol Sante Publiaue 1984: 32: 206-21 I. Rogot E, Goldberg iD. A proposed index for measuring agreement in test-retest studies. J Chron Dis 1966; 19: 991-1006. Walter SD, Irwig LM. Estimation of test error rates, disease prevalence and relative risk from misclassified data: A review. J Clin Epidemiol 1988; 41: 923-937. Walker AM, Velema JP, Robins JM. Analysis of case-control data derived in part from proxy respondents. Am J Epidemiol 1988; 127: 905-914.

Documents

Reliability of environmental and occupational exposure data provided by surrogate respondents in a case-control study of Parkinson's disease