DEVELOPMENTAL REVIEW 1, 322-329 (1981)

The Home Observation for Measurement of the Environment: A Rejoinder to Elardo and

Bradley’s Comment

MARKZIMMERMAN

This paper is a rejoinder to Elardo and Bradley’s comment upon my paper which critiqued their literature review of the research using the Home Observa- tion for Measurement of the Environment (HOME) scale. A summary of both the 16 critical points which I originally raised and their comments on 12 of these issues is presented. Two issues are discussed in some depth: (1) the calculation of the specificity of an instrument, and (2) the reason I included a section on genetic- environment confounding in my initial critique despite the appearance of a simi- larly worded section in Elardo and Bradley’s original review.

A series of articles whose titles include such phrases as “a critique of . . . ) ” “a comment on . . . ,” and “a reply, response or rejoinder to . . .” typically suffer a major problem-the authors often tend to tenaciously defend their point of view or the validity of their research and thereby sidestep, distort, misinterpret, or ignore criticisms of their work. Of course, ofttimes objections are legitimately refuted, but the prevalence of the aforementioned perversions makes it difficult for the reader to draw valid conclusions from such repartees. After having read the three-, four-, or five-article exchange there frequently is still some doubt about where the author finally stands on all of the issues which were raised. For example, if author A does not reply to an objection raised by author B what does author A intend by this nonresponse? Another problem of such debates is that the author with the final reply has his/her concluding statements go unchallenged; consequently, he/she often seems to have presented a stronger position by adeptly (and sometimes accurately) re- futing the preceding criticisms.

With these preliminary comments kept in mind I would like to briefly respond to some of the issues raised by Elardo and Bradley’s (1981b) reply to my critique (Zimmerman, 1981) of their literature review. I have listed in Table 1 the 16 points which I made in my critique. Effort was made to include all of the issues which I originally noted, even those to which Elardo and Bradley subsequently correctly took exception. I have also briefly summarized Elardo and Bradley’s point-by-point responses.

Immediately apparent is that Elardo and Bradley made no comment on 4 of the 16 points (3, 7, 8, 16). Two others (9, 10) were also not discussed because, they asserted, their original literature review (Elardo and Brad-

Requests for reprints should be sent to Dr. Mark Zimmerman, Department of Psychiatry, University of Iowa, 500 Newton Road, Iowa City, Iowa 52242.

322 0273-2297/81/040322-08$02.00/O Copyright @ 1981 by Academic Press, Inc. All rights of reproduction in any foml reserved.

REJOINDER TO ELARDO AND BRADLEY 323

TABLE 1 A POINT-BY-POINT SUMMARY OF ZIMMERMAN’S (1981) CRITICISMS AND RECOMMENDATIONS, AND ELARDO AND BRADLEY’S (1981b) RESPONSES

Issue raised by Zimmerman Elardo and Bradley’s reply

(A) Intc~rru~er Reliahilify 1. Stevenson and Lamb’s (1979) data

were not included in Elardo and Bradley’s calculations of the average percentage agreement between observers.

2. Interrater reliabilities should be re- ported in terms of K coefficients.

3. Interrater reliability estimates may be inflated if the observers are aware that a reliability check is being made.

4. Consensual drift may serve to bias calculations of interrater reliability.

(B) Test -Retest Sruhilir) 5. Test-retest stability refers not only

to reliability coefficients, but also to the stability of the mean scale scores; thus at- tention should be given to both. Barnard and Gortner (1977) and Caldwell and Bradley (1979) both found that with the in- creasing age of the infants the mean HOME scores increased.

6. Wulbert CI al. (1975), in a sample composed of working, middle, and upper- middle classes, reported that the correla- tion between the total HOME scores and SES was .13. The data presented by Ramey et ~1. (1975) indicated that the stan- dard deviations of the HOME scale scores were lower in the middle class sample than in the lower class sample. These studies suggest that a ceiling effect may limit the applicability of the HOME to middle and upper class populations.

(D) Comintcr Validit) 7. Elardo and Bradley’s review omitted

a summary of the quantitative data from

1. Elardo and Bradley still question whether it would be appropriate to include Stevenson and Lamb’s data in the compo tation of the average interobserver agree- ment. However, they suggested that per- haps a comment about Stevenson and Lamb’s discrepant results should have, indeed, been included.

2. Agreed with my recommendation.

3. No comment.

4. Agreed that consensual drift is a problem, but they believed that it is a con- found which “is not likely to have any ap- preciable impact on interrater reliability.”

5. Elardo and Bradley agreed that “per- haps some comment should have been made about the slight increase in HOME scores from 6 to 24 months.” However, they then criticized my “failure to elab- orate on how (one) should deal with the stability of mean scores.”

6. Correctly pointed out my misstate- ment of Stevenson and Lamb’s conclusion regarding the problem of a ceiling effect on the HOME. They criticized my under- standing of the Wulbert PZ a/. study, yet still agreed that a ceiling effect for middle class families exists. However, they noted that the scale “was not intended to dis- criminate between a generally adequate level of parenting and ‘super’ parenting. Consequently, my criticism that the HOME may not be applicable to middle and upper class populations is not a serious problem.

7. No comment.

324 MARK ZIMMERMAN

TABLE l-Continued

Issue raised by Zimmerman

the studies which they presented as pro- viding evidence for the construct validity of the HOME.

Elardo and Bradley’s reply

(E) Predictive Validity 8. The Little Rock longitudinal project

found that HOME scores collected when the child was 24 months old accounted for over half of the variance of the child’s IQ measured at age 3 and 40% of the var- iance of the child’s IQ score assessed at 54 months of age. Additional follow-ups are needed to determine whether the predic- tive validity will decline further.

8. No comment.

9. Direction of effect-Bradley er al. (1979) used a cross-lagged panel analysis in an attempt to determine the causal di- rection of whether the child’s behavior is is responsible for parents creating a partic- ular environment, or whether the environ- ment is of etiological importance in the child’s cognitive development. Their re- sults were inconclusive. Stevenson and Lamb (1979) found only a small relation- ship between HOME scores and infant sociability, thereby suggesting that the en- vironment exerts an effect on the child rather than the reverse.

9 and 10. Believed that their original pa- per had adequately dealt with these two issues; therefore my comments were, in essence, superfluous.

10. Genetic-environment confounding- Willerman (1979) argued that measures of the home environment are not indepen- dent of parent intelligence. Milar et al. (1980) reported that the correlation be- tween HOME total scores and maternal IQ was .60.

(F) Screening and Diagnosis 11. The studies originally reviewed by

Elardo and Bradley provided only circum- stantial evidence regarding the HOME’s efficiency as a screening instrument. Needed are data reporting the sensitivity and specificity of the measure.

11. Agreed that much of the research which they originally reviewed provided only circumstantial evidence attesting to the HOME’s effectiveness as a screening measure.

12. Bradley and Caldwell (1977) incor- 12. Asserted that they made no error in rectly calculated or defined specificity. calculating the specificity of the HOME. They computed the specificity of the They noted, however, that the definition HOME in predicting a child’s IQ at age 3 which they used and the one which I used to be 43%; the corrected figure is 78%. were different.

REJOINDER TO ELARDO AND BRADLEY 325

TABLE I-Continurti

Issue raised by Zimmerman Elardo and Bradley’s reply

13. The recalculated level of specificity is still insufficiently high to justify the use of the HOME as a measure upon which to base referral to intervention programs.

14. Van Doorninck et ~1. (Note 1) com- pared the HOME scale and SES classi- fication in their respective abilities to pre- dict school achievement problems. They found that the overall misclassification rate was lower for the HOME scale (22% vs 36%). Sensitivity was greater using SES classification (89% vs 63%), whereas spec- ificity was higher using the HOME (86% vs 54%).

(G) Matching Environmc~tlts 15. The usefulness of the HOME for

purposes of matching subjects in social science experiments may be limited to lower and lower-middle classes due to a ceiling effect on scores in middle and up- per-middle classes.

(H) Race Differences 16. Elardo and Bradley cited Trotman’s

(1977) study as evidence that the environ- ment is a possible causal factor of racial IQ differences, but they did not cite the crit- icisms of Trotman’s findings made by other investigators.

13. Indicated that my discussion of the implications of the HOME’s low levels of sensitivity and specificity “appear to be equivocal, simplistic, naive, and they also seem to betray a value bias. We have never recommended that the HOME scale be used in isolation as a screening device-nor do we know anyone who would.”

14. Agreed that it is important to com- pare the screening efficiency of the HOME scale and SES meaures. They did not men- tion the Van Doorninck et al. study, but instead stated that “no good direct com- parisons” have as yet been made. They did refer to findings in other studies which suggested the greater screening effective- ness of the HOME scale.

15. Indicated that I “overstated” my case in suggesting that the middle class ceiling effect limits the usefulness of the HOME for purposes of matching subjects in research.

16. No comment.

ley, 1981a) had adequately addressed these issues. Complete or partial agreement is evident on 8 of the remaining 10 points, and total disagree- ment on 2 (13, 15).

With respect to my discussions of the implications of the HOME’s low level of specificity and the limitations of its use for purposes of matching subjects in research, Elardo and Bradley were, perhaps, correct in stating that my comments were somewhat “overstated” and “naive.” However, this does not excuse the absence of a discussion of these issues in their original review.

326 MARK ZIMMERMAN

Another issue unmentioned in their original review was the existence of a ceiling effect for HOME scores in middle and upper-middle class fami- lies. Elardo and Bradley agreed, in fact, that such a phenomenon does occur; however, in spirit, they dismissed its importance. Their rebuttal had some merit.

For the sake of brevity I am not going to discuss each of the objections and suggestions which they made. Rather I will restrict the remainder of this brief paper to two points: one an issue of substance, the other an issue of substance and propriety.

THE CALCULATION OF SPECIFICITY

Elardo and Bradley (1981b) wrote: Zimmerman accuses us of either reporting an error in calculation or using an unconventional definition of specificity. In fact, no error was made in calculating the specificity of the HOME (43%) in the Bradley and Caldwell (1977) article. The definition of specificity used was stated in the article: “the percentage of those classified as having the condition on the basis of their test scores who actually have the condition” (p. 148). The source of the definition, Frankenburg (1973), was also given. Admittedly, the definitions are different. Zimmerman does a service in clarifying the difference.

Elardo and Bradley were correct in stating that no error in calculation was made based upon the definition which they used. Although they noted a difference between the definition of specificity which they used and the one which I used, they failed to acknowledge that theirs is, indeed, not the conventional one used to calculate specificity.

Sensitivity and specificity are epidemiological concepts. In the three epidemiological textbooks which I examined (Friedman, 1974; McMahon and Pugh, 1970; Roberts, 1977) specificity was defined as the proportion of nonindex cases correctly identified. Using the notation in Table 2, specificity is calculated as d/(b + d), and sensitivity as al@ + c). Bradley and Caldwell (1977), using their idiosyncratic definition of specificity, computed specificity as al@ + 6). Based on the data presented in their paper they determined that the specificity of the HOME inventory in identifying low-IQ children was 43%. However, the correct figure is 78%.

TABLE 2 A CONTINGENCYTABLEFORJUDGINGTHEVALIDITYOFA SCREENINGTEST

Results of screening test

Positive

Negative

Actual diagnosis

Disease present Disease absent

a b

C d

REJOINDER TO ELARDO AND BRADLEY 327

As Elardo and Bradley (1981b) noted, the source of the definition, Frankenberg (1973), was given.’ Frankenburg’s definition, however, is identical to the one I described above. “Specificity, the second type of test validity, is the agreement between the test in classifying subjects as negative and the diagnosis in classifying the same subjects as non- diseased. . . . Specificity is calculated by the formula: dl(b + d) x 100” (P. 30).

GENETIC-ENVIRONMENT CONFOUNDING AND DIRECTION OF EFFECT

The other point of consideration is more difficult to deal with because it is not solely an issue of substance, but also one of propriety. Elardo and Bradley (1981b) wrote:

Zimmerman stated that there are two major problems which should prevent the uncritical acceptance of the findings of high predictive validity of the HOME scale: (1) direction of effect, and (2) the compounding of genetic and environmental ef- fects. Actually, we do not see why it was necessary to raise this issue in his critique, since we feel we adequately did so in our original paper. In fact, a signifi- cant portion of his discussion of Willerman (1979) was a direct quote (which was not indicated by Zimmerman) or a close paraphrase of our discussion.

I wish to defend myself against their charge that I, in essence, incorpo- rated a section of their paper into my own without recognizing their con- tribution. Much of the discussion of Willerman which appeared in Elardo and Bradley (1981a) and Zimmerman (1981) was very similar to a discus- sion of this material that I had written prior to Elardo and Bradley (1981a). The original draft of Zimmerman (1981) had a footnote to this effect, which I later decided to omit.

The second, substantive, reason for “readdressing” the issues of genetic-environment confounding and direction of effect in my critique was that I did not believe that Elardo and Bradley had, in fact, adequately dealt with the issues in their review article. For example, they did not report that Milar, Schroeder, Mushak, Dolcourt, and Grant (1980) found a correlation of .60 between maternal IQ and HOME scores. Nor did Elardo and Bradley discuss the implication of Stevenson and Lamb’s (1979) data which showed a minimal association between HOME scores and a measure of infant sociability. I believe that these are noteworthy omissions, and therefore the reappraisal of these two issues was justified.

’ The reader should note that the reference given by Bradley and Caldwell (1977) and Elardo and Bradley (1981b) for the book containing Frankenburg’s paper is incorrect. See the bibliography of this paper for the correct citation.

328 MARK ZIMMERMAN

CONCLUSION

An overview of research employing an assessment measure should provide a balanced, critical, and comprehensive examination of the rele- vant empirical and conceptual issues. In my critique of Elardo and Brad- ley’s literature review 16 points were noted which I believed had not been adequately discussed. Elardo and Bradley’s reply addressed only 12 of these issues. They agreed, wholly or in part, with many of my objections. Notably, some of their criticisms of my critique were well-taken and have prompted a reevaluation of those issues on my part. Nevertheless, some differences remain. I hope this paper has served to clarify these differ- ences,

REFERENCES Barnard, K. E., & Gortner, S. R. Child health assessment: Part Two-Resrrlts of the first

twelve months DHEW Pub. No. (HRA) 75-30. Bethesda, Md.: U.S. Department of Health, Education, and Welfare, Public Health Services, Health Resources Adminis- tration, Bureau of Health Resources Development, Division of Nursing, 1977.

Bradley, R. H., & Caldwell, B. M. Home Observation for Measurement of the Environ- ment: A validation study of screening efficiency. American Journal of Mental Defy- ciency. 1977, 81, 417-420

Bradley, R. H., Caldwell, B. M., & Elardo, R. Home environment and cognitive develop- ment in the first 2 years: A cross-lagged panel analysis. Developmental Psychology, 1979, 15, 246-250

Caldwell, B. M., & Bradley, R. H. The Home Observation for Measurement of the Envi- ronment. Little Rock: Univ. of Arkansas Press, 1979.

Elardo, R., & Bradley, R. H. The Home Observation for Measurement of the Environment (HOME) scale: A review of research. Developmental Review, 1981, 1, 113-145. (a)

Elardo, R., & Bradley, R. H. The Home Observation for Measurement of the Environment: A comment on Zimmerman’s critique. Developmental Review, 1981, 1, 314-321. (b)

Frankenburg, W. K. Increasing the lead time for the preschool and handicapped child. In J. B. Jordan & R. F. Dailey (Eds.), Invisible College Conference on Early Childhood Education and the Exceptional Child. SatI Antonio. Tex. 1972. Not All Little Wagons are Red. Arlington, Va.: Council for Exceptional Children, 1973.

Friedman, G. D. Primer of epidemiology. New York: McGraw-Hill, 1974. MacMahon, B., & Pugh, T. F. Epidemiology: Principles and methods. Boston: Little,

Brown, 1970. Milar, C. R., Schroeder, S. R., Mushak, P., Dolcourt, J. L., & Grant, L. D. Contributions

of the caregiving environment to increased lead burden of children. American Journal of Mental Deficiency, 1980, 14, 339-344

Ramey, C. T., Mills, P., Campbell, F. A., & O’Brien, C. Infants’ home environments: A comparison of high-risk families and families from the general population. American Journal of Mental Dejkiet1c.v. 1975, 80, 40-42.

Roberts, C. J. Epidemiology for clinicians. Tunbridge Wells, Kent: Pitman, 1977. Stevenson, M. B., & Lamb, M. E. Effects of infant sociability and the caretaking environ-

ment on infantile cognitive performance. Child Development, 1979, 50, 340-349 Trotman, F. K., Race, IQ, and the middle class. Journal of Educational Psycholog.v, 1977,

69, 266- 273 Willerman, L. Effects of families on intellectual development. American Psychologist. 1979,

34, 923-929

REJOINDER TO ELARDO AND BRADLEY 329

Wulbert, M., Inglis, S., Kriegsmann, E., & Mills, B. Language delay and associated mother-child interactions. Devrloprnentul Psychology. 1975, 11, 61-70.

Zimmerman, M. The Home Observation for Measurement of the Environment: A comment on Elardo and Bradley’s review. Develupmetzrul Review, 1981, 1, 301-313.

REFERENCE NOTE 1. Van Doorninck, W. J., Caldwell, B. M., Wright, C., & Frankenburg, W. K. The In-

l’rntory of Home Stimulation as (I predictor of school comprtencc. Paper presented at the Society for Research in Child Development, Denver, 1975.