DOCUMENT RESUME

ED 323 229 TM 015 432

AUTHOR Olejnik, Stephen F.; Algina, JamesTITLE Type I Error Rate anu Power of Rank Transform ANOVA

When Populations Are Non-Normal and Have EqualVariance.

PUB DATE 85NOTE 23p.

PUB TYPE Journal Articles (080) -- Reports -Research/Technical (143)

JOURNAL CIT Plorida Journal of Educational Research; v27 nlp61-81 Fall 1985

EDRS PRICE MF01/PC01 Plus Postage.DESCRIPTORS *Analysis of Variance; Computer Simulation; *Error

Patterns; *Population Distribution; Sample SizeIDENTIFIERS *Power (Statistics); Rank Order Transformation; *Type

I Errors; Variance (Statistical)

ABSTRACT

The rank transformation approach to analysis ofvariance (ANOVA) as a solution to the Behrens-Fisher problem wasexamined. Using simulation methodology, four parameters weremanipulated for the two-group design: (1) ratio of populationvariances; (2) distribution form; (3) sample size; and (4) populationmean difference. As a general solution to the group varianceinequality problem, the results of thi- study do not providesufficient evidence to recommend any single analysis approach. Whilethe rank transform approach was less sensitive to variance inequalitythan was the parametric ANOVA F-ratio, unacceptably high Type 1 errorrates were obtained when cell frequencies and group variances wereinversely related. With equal cell frequencies or when cellfrequencies we:e directly related to group variances, appropriateType I error rates were obtained. Under these conditions, the

Brown-Forsythe prccedure for comparing group means provided greaterpower except wnen the sampled distribution was leptokurtic. It iscontended that before computing hypothesis tests, researchers shouldfirst obtain descriptive summary statistics to determine the sampledistribution characteristics and to use this information to guidetheir choice of analysis procedures. Five ta es present simulationdata. (Author/SLD)

*******X*****************V********************V************************ Reproductions supplied by EDRS are the best that can ta made ** from the original document. *

***********************************************************************

a

U S DEPARTMENT OF EDUCATIONOthce et EnuCal,Onat ReSearCh and ImprO,ernent

EDUCATIONAL RESOURCES INFORMATIONCENTER IERICI

Fi'TIIrus clocument has been reproduced as,ece.ved from Ine Delson or eicanizabonortgtnafieg

C Mnor oaedes Nave ("ten maCle to mrnpovereprOductmon Ouahty

Po.nts ,,,ew OOp.n.or5 stated,e !Ns docu-ment do not ,eCessanly epresent ottmc atOE RI cos,n,0n 0, 00 cy

"PERMISSION TO REPRODUCE THISMATERIAI. HAS BEEN GRANTED BY

L-ou be&I

TO THE EDUCAIIONAL RESOURCESINFORMATION CENTER (ERIC)"

TYPE I ERROR RATE AND POWER OF RNNK TRANSFORMANOVA WHEN POPULATIONS ARE NON-NORMAL AND

HAVE EQUAL VARIANCE

Stephen F. OlejnikUniversity of Georgia

and

James AlginaUniversity of Florida

BEST COPY AVAILABLE

Florida Journal ofEducational ResearchFaZZ 1985, Vol. 27,

No. 1, Pp. 61-82

Type I Error Rate and Power of Rank TransformANOVA When Populations are Non-Normal and

Have Equal Variance

Stephen F. OlejnikUniversity of Georgia

and

James AlginaUniversity of Florida

ABSTRACT. The rank transformation approach toanalysis of variance as a solution to theBehrens-Fisher problem is examined. Usingsimulation methodology four parameters weremanipulated for the two group design: (1)ratio of population variances; (2) distri-bution form; (3) sample size and (4) popula-tion mean difference. The results indicatedthat while the rank transform approach was

less sensitive to variance inequality than theparametric ANOVA F-ratio, unacceptably highType I error rates were obtained when cellfrequencies and group variances were inverselyrelated. With equal cell frequencies and/orwhen cell frequencies were directly related togroup variances, appropriate Type I errorrates were obtained. Under these conditionshowever, the Brown-Forsythe procedure forcomparing, group means provided greater powerexcept when the sampled distribution wasleptokurtic.

Both empirical and analytic studies have repeatedlyshown that paramet:ic analysis procedures forcomparing group means are extremely sensitive topopulation voriance inequality when sample sizes are

Zr.

.4')

markedly unequal. Wben sample size and group variance

61

V-3

Olejnik and Algina

are positively correlated, the nominal significancelevel is underestimated while with a negativerelationship between sample size and group variant,the nominal significance level is overestimated(Glass, Peckham and Sanders, 1972). Even with equalsample sizes, Ramsey (1980) has shown that the actualprobability of a Type I error for the t-test mayeither over or underestimate the nominal significancelevel. Developing alternhtive data analysisstrategies when population variances differ, alsoknown as the Behrens-Fisher problem, has thereforebeen an area of considerable interest, and severalsolutions to the problem have been suggested. Theprocedure offer'd by Welch (1947, 1951), inparticular, has gained considerable attention.Welch's solution modifies the ANOVA F-ratio byweighting the sample means by the ratio of the groupfrequency to the group variance. In addition thedegrees of freedom error are adjusted so that thecomputed statistic approximates the F distribution.Wang (1971) has shown that this approximation issatisfactory for most situations. James (1951)suggested a similar weighting procedure but used thechi-square distribution as the reference distribution.Recently Brown and Forsythe (1974a) have suggested a

slightly different approach to the Behrens-Fisherproblem. Their statistic takes the ratio of the sumsof squares between groups to a weighted sum of groupvariances. The test statistic has an approximated Fdistribution. For the two group case, the Welch andthe Brown-Forsythe procedures are identical(Brown-Forsythe, 1974a), but differ when multiplegroups are compared. Both procedures have beengeneralized for factorial designs (Brown and Forsythe,1974b; Johansen,.1980; Algina and Olejnik..1984).A number of investigations have studied both of

these strategies and compared them with respect totheir Type I error rates and statistical power. Theresults of these studies have shown that bothapproaches are insensitive to variance inequality whenthe sampled distributions were normal (Kohr and Games,1974; Brown and Forsythe, 1974a; Levy, 1978; Dijkstra.and Werter, 1981; Lee and Fung, 1983). Undernon-normal parent distributions the Brown-Forsythe

62

4

Rank Transform ANOVA

statistic was shown to provide appropriate Type Ierror rates (Clinch and Keselman, 1982; Lee and Fung,1983). The results with Welch's approach however havebeen mixed and inconsistent. There is some evidenceto indicate that the approach is liberal for skeweddistributions when the number of levels of thegrouping factor is four or more (Clinch and Keselman,1982; Levy, 1978). Levy on the other hand foundappropriate Type I error rates when there were threelevels of the independent variable and the sampledpopulation had a chi-square distribution. When datawere sampled from heavy-tailed distributions, someresults have indicated that Welch's procedure providesa conservative test of group means (Yuen, 1974; Leeand Fung, 1983). Other evidence however indicatesappropriate Type I error rates (Clinch and Keselman,1982). Differences in these conclusions may be afunction of the degree to which the sampledpopulations departed from normality. Fthally forlight-tailed distributions, the Welch procedure hasbeen shown to provide a liberal test of means (Levy,1978), but other results indicate that appropriateType I error rates are possible (Yuen, 1974).When the procedures were compared in terms of tneir

statistical power, the results have been mixed butgenerally consistent. Both procedures providecomparable power when the population distributions arenormal and the variances are equal. Under thiscondition both procedures are only slightly lesspowerful than the ANOVA F-ratio (Brown and Forsythe,1974a; Dijkstra and Werter, 1981; Clinch and Keselman,1982; Lee and Fung, 1983). With unequal variances anda normal parent distribution, the Brown-Forsytheapproach provided greater power when the extreme meanhad lower variance, while the Welch procedure was moresensitive if the extreme mean had the large variance(Brawn and Forsythe, 1974a; Dijkstra and Werter, 1981;Lee and Fung, 1983). Clinch and Keselman howeverfound very little difference in statistical powerbetween the procedures for this condition. Thestatistical power for all of the procedures studiedhowever was relatively low, and that may explain theinconsistency in the results reported by Clinch andeselman. Finally for heavy-tailed distributions the

63

Olejnik and AZgina

Welch procedure provided a slight power advantage- (Clinch and Keselman, 1982; Lee and Fung, 1983).

Recently, Dauphin (1983) considered a differentapproach to the Behrens-Fisher problem. She suggestedtransforming the original data by using rarks beforegroup means are compared. After ranking the data fromhighest io lowest across all comparison groups, theparametric analysis of variance F-ratio is computed.This strategy of transforming data using ranks beforecomputing parametric analysd's has been suggested byConover and Iman (1981) as a linking procedure betweenparametric and nonparametric analysis strategies.They have suggested that the ranking approach can beused in a variety of research contexts andconsiderable research has keen conducted using thisapproach generally with positive results. Nath andDuran (1981a, 1981b) studied the procedure when twogroup means are to be compared, Conover and Iman(1982) applied the approach to analysis of covariance,Iman and Conover (1979) used the rank transformationin a regression problem, and Iman (1974) studied theapproach for factorial designs when an interaction waspresent. Although the theoretical rationale for theprocedure is not fully developed, progress in thatdirection has been reported by Iman, Bora and Conover(1984).

The use of the rank transformation has beenmotivated primarily as an alternative analysisstrategy to parametric statistics when sampleddistributions were non-normal. In this context therank transformation has often provided a moresensitive test of the location parameter than theparametric alternative. The rationale of applying therank transformation as a solution to theBehrens-Fisher problem was based on previous findingsthat nonparametric strategies, while affected byvarianta inequality, are less sensitive than theparametric alternatives (Wetherill, 1960). Glazer(1963), for example, empirically demonstratedWetherill's asymptotic results showing that theWilcoxOn-Mann-Whitney probability of a Type I errorwas less affected than Student's t-test for independ-ent sample means when the population variancesdiffered. Since the rank transform is monotonically

64

Rank Transform ANOVA

related to the Wilcoxon test, Dauphin expected similarconclusions. Her results confirmed her expectationshowing that the actual Type I error rate for the ranktransform did not deviate greatly from the nominalsignificance level when the sampled population wasnormal.

Since the rank transformation has gainedconsiderable interest and Dauphin's results indicatethat the approach may have some merit in somesituations, it was decided to examine this proposedsolution to the Behrens-Fisher problem a littlecloser. Specifically the purpose of the study was toanalyze the empirical Type I error rates of the ranktransform ANOVA with parametric analysis of varianceand Brown-Forsythe's procedure when populationvariances differed, and the distributions were normalor non-normal. In addition, for those situationswhere appropriate Type I error rates were observed,the statistical power estimates for small, medium andlarge differences in group means were compared.

Computer Simulation

In order to calculate empirical Type I error ratesand statistical power estimates for .each of thecompeting analysis strategies under a variety cfconditions, four factors were manipulated: .1) samplesize; 2) distribution form; 3) population meandifference and 4) population variance inequality.Although all three of the procedures can be used forcomparing group means in multiple group designs,including factorial designs, the present investigationwas limited to comparisons between two groups.

Sample Size. Samples of (10,15), (15,10), (20,20),(17,23), and (23,17) were included in theinvestigation. The sample sizes consideree here werethought to be moderate and representative of thoseoften found in research studies in the socialsciences. Small departures from equal n's were chosento represent common attrition rates in socialresearch.

Distribution Form. A normal and four non-normalparent distributions were considered. The non-normaldistributions included a light-tailed, platykurttc

65

Olejnik and Algina

distribution, a symmetric, leptokurtic (heavytailed)distribution, a moderately skewed distribution, and adistribution which was both skewed and leptokurtic.The population cnaracteristics of these distributionsare discussed in the data generatj.on section.

Population Mean Difference. To study the Type Ierror rates of the three procedures, data weregenerated from populations which had a common mean.Power estimaees were obtained by comparing theproportion of hypotheses rejected when data weresampled from populations which differed by .2, .5, or.8 pooled standard deviation units. These differencesin the location parameters have been suggested byCohen (1977) as representing small, medium, and largeeffects respectively.

Population Variances- The present Etudy consideredpopulations which had common variances as well as sixlevels of variance inequality. Specifically data weregenerated from populations with the f011owing variancepairs: (1,1), (1,1.5), (1,2.0), (1,2.5), (1,3.0),(1,3.5), and (1,4.0). The choice of these variancedifferences was based on two considerations. First, itwas believed that'the conditions considered reflectedcommon situations encountered by applied researchers.Second, it was believed that with the unequalsample size combinations studied, the variancedifferences would affect the Type I error rate of theparametric ANOVA Fratfo.

Data Generation. Data for the study were generated'using the SAS computing package. Scores on thedependent measure were created based on the linearmodel function Yij = i .. +a .j +a j cij, where Yij isthe ith observation in the jth group. The grand meanp .. was set equal to 10. The effect size parameterfor the jth group, a i, was varied from 0, .2, .5, or.8 pooled standard deviation units to study the effectpopulation mean difference. In all cases the shiftparameter was added to the second group so that111 < P 2 The random error component c ij was

generated using the SAS NORMAL function to simulatescores, Xij, from a standard normal distribution. Fora normally distributed error component, c ij was setequal to Xij. For a nonnormally distributedcomponent, Xij was transformed using a power function

66

Rank Transform ANOVA

developed by Fleishman (1978): eij

[(dXij+c)Xij+b]Xij-s-a. The constants a, b, c and dwere chosen to transform the standard normal variableto a variable with known skewness and kurtosis andnull mean and unit variance. Four nonnormaldistributions we::e considered in the study.Descriptive statistics and frequency distributions athalf standard deviation intervals are included inTable 1. Values reported in the table are based on20,000 random variables generated for each

Th'e variance of the observations ingroup one was kept constant at 1 for all conditionsstudied while the variance of the second group wasincreased from 1 to 4 in increments of .5 units bymultiplying the random error component by the desiredstandard deviation.

Computed Test Statistics. In each sample generated,the group means were compared using the parametricanalysis of variance Fratio, the BrownForsythe(1974a) test statistic, and the rank transformanalysis of variance Fratio.

The parametric analysis of variance Fratio iscomputed as the ratio of the mean square between groupmeans to the pooled within group variance:

E n (F. ....Y..)2/(J-1)

E (n 1)s2/(NJ)

where nj is the number of observations in the jthgroup; N is thc total number of observations in thesample; J is the number of groups in the study; Y.j isthe sample mean for the jth group; Y.. is the grandmean; S5 is the variance of the jth group. Thecritical test statistic is obtained from the Fdistribution with J-1 and NJ degrees of freedom.The rank transform ANOVA Fratio is computed using

the same ...ormula as parametric ANOVA with thedependent variable obtained by replacing the originalobservations with the rank of the observation. Theobservations are ranked by assigning a 1 to the lowestobservation and N to the highest observation in the

67

Olejnik and Algina

Table 1

Frequency Distributions and Descriptive Statistics

Distributions

Interval

.'

Normal Platykurtic Skewed LeptokurticSkewed/

Leptokurtic

- w ,-3.0

-3.0,-2.5-2.5,-2.0

17

85

232

151

119301

-2.0,-1.5 889 1552 601-1.5,-1.0 1885 2297 3605 1257-1.0,-0.5 2470 2917 3976 2816 8555-0.5, 0.0 3826 3235 3591 4745 42190.0, 0.5 3817 3177 2053 4753 25770.5, 1.0 3038 2805 2345 2748 17771.0, 1.5 1849 2411 1552 . 1343 11421.5, 2.0 855 1606 1039 586 6712.0, 2.5 332 520 .263 4402.5, 3.0 86 230 178 268 .3.0, w 19 89 139 351

Mean -.0015 .0049 .0009 .0004 - .0063Variance .9836 1.0109 1.0631 1.0292 .9774Skewness .0004 - .0005 .7266 - .1297 1.6820Kurtosis -.0938 -1.0131 - .0846 3.5547 3.1517

NI68

Rank Transform ANOVA

total sample across all groups. If ties are presentthe average rank is assigned to all tied observations.The F-ratio is computed as:

Enj(

FR

j

E(n -1)S2 /N-Jj R

where is rhe mean rank of group j; IL. is thegrand mean rank N4-1/2 ; S2Rj is the variance on theranks for the jth group. The critical test statisticfor the rank transform F-ratio is the same as that forthe parametric ANOVA.The Brown-Forsythe statistic is obtained as the

ratio of the sum of squares between groups and aweighted sum of within group variance:

FBF

I it

EniiE(1- 4S2

N jjwhere the terms are defined as stated above. TheBrown-Forsythe F statistic has an F distribution withdegrees of freedom J-1 and f where 1/f is equal to

[c2/(n -1)] and c =(1-N121)S2/ d(1- 1-1.+1-)S2 ]

j j N j

For each condition, 1000 replications of the threestatistics were computed, and the frequency at whicheach procedure rejected the null hypothesis of equalpopulation means at the .05 level was recorded. Inevaluating the robustness of each procedure, it wasdecided that observed proportions of Type I errors twostandard errors above or below the nominalsignificance level would be judged as unacceptable.Based on 1000 replications, observed Type I errorrates outside the interval (.036, .064) wereconsidered nonrobust.

11 69

Olejnik and Algina

Results

The results of the study are reported in twosections. In the first section the empirical Type Ierror rates for the BrownForsythe, the parametricANOVA, and the rank tranaform ANOVA are presented forincreasing variance inequality with equal and unequalsample size combinations. The second section presentsthe proportions of hypotheses rejected when populationdiffered by .2, .5, or .8 pooled standard deviationunits representing small, medium, and large effectsizes. The power results are reported only for thoseconditicns where appropriate Type I error rates wereobtained.

Type I Error Rates

As a preliminary test of the computer program andthe data generation procedure, data for three samplesize combinations were generated from populationsidentical in their form, scale, and location. Thesample means for these samples were compared using thethree analysis procedures under consideration, and theproportion of hypotheses rejected at a nominalsignificance level of .05 were recorded. Table 2reports the results of these analyses for the fivedistribution forms studied. None of the observedproportions exceeded two standard errors above orbelow the expected five percent level. These resultstherefore support the adequacy of the data generationand analysis procedures used in the study.The observed Type I error rates, as the difference

in group variances increased, are reported in Table 3for the five distribution forms with equal and unequalsample size combinations. For the normal distributionthe results reported here are consistent with thosepresented by Dauphin (1983). The Type I error ratefor the rank transform AMATA was affected to a lesserdegree than the parametric ANOVA Fratio. However, forsituations where the smaller samples had greatervariance, the proportion of Type I errors were morethan two standard errors above the nominalsignificance level and therefore judged as beingunacceptably high. When large samples had greater

70

1 2

Rank Transform ANOVA

Type I Error Rates for the Brown-Forsythe (BF),Parametric ANOVA (F), and the Rank Transform (RF)ANOVA

Sanple Size

Distribution

15/10

RF BF

20/20

RF BF

23/17

PFBF F F F

Norma1 .044 .046 .050 444 .044 .043 .060 .064 .059

Platykurtic .057 .058 .058 .056 .056 .052 .054 .051 .048

Skewed .050 .043 .050 .r)52 .052 .055 .055 .057 .054

Leptokurtic .048 .051 .056 .041 .043 .046 .058 .056 .061

Skewed and .045 .045 .053 .046 .048 .052 .044 .040 .045

Leptokurtic

Note: Nominal .05.

1 3

71

Table

Type 1 Error Retes for the Brown-Forsythe- (BF), Parametric ANOVA (F), and theRank Transform '40VA (RF)

Sample Size (n1/n2)

Variance 10/15 15110 20/20 17/23 23/17Ratio

Distribution o2:n

2BF F RF BF F RF BF F RF BF F RF BF F RF1 2

Normal

1,1,1E0cm-tic

Skewed

1:1.5 .059 .049 .052 .050 .069 .066 .044 .045 .C42 .047 .038 .050 .046 .051 .0521:2.0 .05 .041 .055 .055 .069 .066 .054 .057 .066 .057 .044 .042 .037 .045 .0461:2.5 .056 .040 .043 .048 .067 .G68 .049 .049 .056 .046 .636 .047 .053 .068 .0691:3.0 .055 .034 .045 .053 .089 .079 .050 .052 .061 .044 .037 .045 .042. .060 .0561:3.5 .049 .029 .037 .055 .085 .075 .046 .046 .049 .051 .034 .040 .063 .078 .0741:4.0 .044 .010 .041 .043 .084 .081 .067 .071 .000 .047 .034 .047 .043 .060 .r67

1:1.5 .043 .040 .047 .U40 .063 .068 .043 .043 .043 .044 .041 .048 .048 .056 .0551:2.0 .040 .028 .034 .050 .061 .066 .048 .048 .054 .045 .040 .044 .048 .056 .0601:2.5 .039 .024 .034 .060 .079 .078 .053 .055 .058 .053 .039 .047 .058 .073 .0671:3.0 .058 .040 .059 .052 .076 .075 .059 .059 .062 .055 .038 .046 .045 .066 .0721:3.5 .045 .027 .040 .052 .077 .078 .049 .051 .061 .056 .041 .058 .049 .081 .0741:4.0 .055 .035 .045 .059 .087 .094 .052 .053 .063 .050 .033 .051 .G53 .078 .077

1:1.5 .038 .039 .052 .060 .059 .069 .057 .057 .057 .048 .045 .055 .053 .060 .0711:2.0 .057 .045 .054 .054 .078 .090 .041 .041 .056 .055 .043 .060 .051 .057 .0751:2.5 .054 .049 .071 .065 .979 .089 .046 .047 .073 .057 .047 .072 .053 .072 .0931:I.0 .051 .035 .052 .057 .089 .088 .048 .049 .080 .047 .036 .065 .053 .070 .0811:35 .049 .035 .064 .048 .075 .098 .049 .050 .079 .063 .047 .086 .041 .068 .0971:4.0 .045 .027 .053 .054 .088 ..109 .053 .056 .079 .057 .040 .081 .059 077 .110

1 4

Leptokurtlk: 1:1.5 .034 .027 .018 .049 .068. .061 .052 .052 .049 .053 .047 .054 .045 .057 .0491:2.0 .053 .042 .050 .036 .053 .045 .043 .045 .053 .047 .035 .050 .053 .067 .06212.5 .046 .028 .043 .019 .070 .063 .051 .051 .048 .064 .047 .053 .049 .065 .0531:3.0 .046 .028 .038 .041 .073 .052 .050 .051 .057 .049 .040 .054 .050 .072 .0691:3.5 .036 .025 .037 .051 .088 .077 .04S .047 .048 .044 .029 .051 .061 .086 .0711:4.0 .040 .022 .048 .041 .080 .073 .037 .039 .042 .064 .039 .055 .051 .082 .079

Skewed and 1:1.5 .039 .040 .067 .068 .070 .103 .058 .059 .108 .053 .047 .089 .046 .055 .097leptokurtie 1:2.0 .046 .048 .095 .056 .062 .120 .043 .047 .119 .056 .052 .145 .055 .061 .149

1:2.5 .050 .049 .116 .067 .077 .147 .668 .069 .160 .039 .036 .137 .068 .073 .1821:3.0 .045 .043 .118 .071 .035 .150 .060 .060 .190 .042 .035 .155 .071 .079 .1881:3.5 .045 .939 .110 .077 .096 .170 .058 .061 .192 .045 .038 .203 .077 .095 .2091:4.0 .056 .049 .149 .076 100 .175 .059 .062 .229 .066 .047 .192 .068 .089 .221

Note: Umalnal n .05

15

Olejnik and Algina

variance the rank transform had acceptable Type Ierror rates while the ANOVA F-ratio underestimated thenominal significance level. With equal sample sizes,both the ANOVA F and the rank transform were notseriously affected by variance inequality. TheBrown-Forsythe procedure provided appropriate Type Ierror rates for all degrees of variance inequality andsample size combinations.With symmetric, non-normal distributions the

observed Type I error rates were similar to thoseobtained under' the normal populations. The ranktransform ANOVA had Type I error rates which wereaffected to a lesser degree than the parametric ANOVAF-ratio. Error rates within the acceptable range wereobtained for the rank transform approach when samplesizes were equal and when the larger sample size hadgreater variance. When the sample with fewerobservations had greater variance, the observed Type Ierror rate exceeded the nominal significance level bymore than two standard errors. When samples wereselected from skewed ppulations, the rank transformapproach had observed Type I error ratesoverestimating the nowlnal significance level for allsample size combinations except when samples of(10,15) were selected. Under the latter condition,appropriate Type I error rates were obtained. TheType I error rates for the ANOVA F-ratio were asexpected and were similar to those obtained undernormal distributions. With the skewed and leptokurticdistribution, the rank transform became quite liberal,even for the condition with sample sf.zes of (10,15).Again Type I error rates for the parametric F weresimilar to those obtained with the normaldistribution. The Brown-Forsythe procedureoverestimated the nominal significance level when thesamples distribution was both skewed and leptokurticand the larger variance was matched with the sampleshaving fewer observations. When larger samples werematched with lower variance, appropriate Type I errorrates were obtained. These results were consistentwith those reported by Clinch and Keselman (1982) intheir analysis of Welch's procedure.In summarizing these results, the observed Type I

error rates for the rank transform was affected to a

74

Rank Transform ANOVA

lesser degree than the parameeric ANOVA F-ratio whenthe samples distributions were symmetric. Thidconclusion is consistent with that predicted byWetherill (1960) and previously demonstrated for the:ormal distribution by Dauphin (1983). With sLeweddistributions however, the observed Type I error rateoveresttnated the nominal significance level. Theeffect of variance inequality on the statistical rowerof the rank transform ANOVA when the sam'leddistribution was symmetric is presented in the nextsection.

Statistical Power

The proportion of hypotheses rejected when thepopulations differed by a small, medium, or largeeffect size (.2, .5, or.8 pooled standard deviationunits respectively) are reported in Tables 4 and 5 forths symmetric distributions studied when samples wete(20,20) and (17,23) respectively. With equal samplesizes the Brown-Forsythe and parametric ANOVA providedcomparable power estimates for all three symmetricdistributions. When sample sizes were unequal theBrown-Forsythe procedure provided a more sensitivetest for the difference in population means for allthree distributions. These results were expectedsince 'under the conditions studied with unequal samplesizes, the F-ratio leads to a conservative test.

Differences between power estimates for the ranktransform ANOVA and those provided by theBrown-Forsythe and the parametric ANOVA procedureswere similar when sample sizes were equal or unequal.For the normal and platykurtic distributions, the ranktransform ANOVA provided power estimates slightlylower than those of the other two procedures. Whenthe sampled distribution was leptokurtic, however, therank transform procedure provided a more sensitivetest for the difference in population means thaneither the Brown-Forsythe or the parametric ANOVA.

Conclusions

The results of the study indicate that the ranktransformation approach to analysis of variance can

75

r7

Olejnik and Algina

Table 4

Proportion of Hypotheses Rejected for theBrown-Forsythe (BF), Parametric ANOVA (F) and the RankTransform ANOVA (RF)

VarianceRatio

Distribution

Normal Platykurtic Leptaurtic

ZffectSize

2:

2BF F RFal ac

Small 1:1.0 .098 .098 .0901:1.5 .096 .099 .1031:2.0 .095 .099 .0961:2.5 .087 .092 .0891:3.0 .101 .102 .1091:3.5 .091 .092 .0991:4.0 .094 .084 .090

hediun 1:1.0 .377 .378 .3421:1.5 .341 .342 .3191:2.0 .335 .344 .3241.:2.5 .335 .338 .3301:3.0 .357 .363 .3511:3.5 .318 .326 .3121:4.0 .364 .369 .349

Large 1:1.0 .718 .718 .6781:1.5 .696 .697 .6621:2.0 .678 .682 .6441:2.5 .696 .702 .6561:3.0 .682 .690 .6371:3.5 .683 .689 .6611:4.0 .666 .673 .641

BF F RF Sf F RF

.100 .101 .089 .100 .100 .115

.095 .096 .089 .065 .067 .091

.084 .086 .078 .099 .100 .106

.109 .110 .097 .094 .097 .096

.096 .099 .088 .107 .110 .131

.102 .104 .095 .095 .097 .118

.097 .101 .084 .110 .115 .109

.304 .304 .265 .376 .378 .417

.328 .329 .287 .348 .351 .388.324 325 .275 .364 .364 .410.326 .330 .269 .336 .339 .408.345 .348 .281 .334 .344 .405.298 .302 .236 .370 .373 .443.341 .348 .273 .343 .352 .441

.699 .700 .640 .697 .701 .766.705 .707 .623 .725 .728 .806.689 .694 .611 .719 .724 .798.668 .676 .554 .696 .703 .776.671 .678 .567 .697 .703 .786.655 .663 .537 .706 .721 .800.689 .694 .553 .689 .697 .779

Note: 112 n2 20

76 18

Rank Transform ANOVA

Table 5

Proportion of Hypotheses Rejected for the

Brown-Forsythe (BF), Parametric ANOVA F, and the RankTransform ANOVA (RF)

EffectSize

VarianceRatio Normal

2 2C 1:02 BF F RF

Small 1:1.0 .097 .095 .090

1:1.5 .115 .108 .1101:2.0 .104 .089 .083

1:2.5 .095 .070 .086

1:3.0 .089 .067 .083

1:3.5 .102 .074 .088

1:4.0 .101 .085 .091

Medimm 1:1.0 .350 .342 .327

1:1.5 .331 .314 .304

1:2.0 .384 .355 .339

1:2.5 .365 .321 .326

1:3.0 .377 '.324 .348

1:3.5 .383 .324 .3491:4.0 .366 .295 .309

Large 1:1.0 .666 .672 .625

1:1.5 .707 .680 .662

1:2.0 .742 .700 .699

1:2.5 .738 .688 .687

1:3.0 .754 .679 .689

1:3.5 .743 .688 .675

1:4.0 .772 .703 .712

Note: n1

17' 2n 23

Distribution

Platvkurtic Leptokurcic

BF

.082

.087

.093

.080

,102

.104

.101

.304

.356

.358

.365

.350

.356

.371

.693

.691

.711

.742

.737

.736

.741

F RF SF F RF

.082 .082 .078 .078 .794.078 .077 .080 .072 .097

.076 .086 .098 .0E2 .120

.064 .071 .100 .077 .117

.084 .091 .125 .099 .143

.079 .089 .119 .091 .112

.073 .075 .112 .088 .113

.310 .281 .369 .372 .420

.329 .296 .382 .367 .429

.314 .278 .365 .247 .417

.327 .284 .389 .344 .437

.299 .252 .389 .337 .448

.301 .270 .390 .345 .448

.294 .260 .411 .346 .448

.698 .637 .671 .674 .762

.669 .617 .731 .718 .792

.683 .620 .753 .712 .805

.698 .610 .753 .715 .828

.685 .610 .772 .714 .819

.679 .598 .740 .700 .790

.688 .592 .761 .709 .834

19

77

Olejnik and Algina

provide a solution to the BehrensFisher problem, butthis solution is appropriate only for a limited set ofconditions. In particular the rank transform ANOVAmay be recommended when sample frequencies arepositively related to group variances and the form ofthe population distribution is leptokurtic. Underthat condition the actual Type I error rate does notoverestimate the nominal significance level and therank transform provides a slight power advantage overthe BrownForsythe solution. This result wasinteresting in that the power advantage for the ranktransform procedure was obtained even though theactual T-,-pe I error rate underestimated slightly thenominal significance level. These results indicatethat Type I error rate alone should not be used toevaluate or compare .statistical analysis strategies.On the other hand, consideration of both statisticalpower and actual Type I error rates do provide minimumcriteria in judging the usefulness of analysisalternatives.

For other symmetric distributions the rank transformprocedure did not provide any statistical advantagecompared to the BrownForsythe procedure. With skewedpopulation distributions, however, the rank transformapproach overestimated the nominal significance leveleven when the sample frequencies were equal. Thisfinding may be viewed as an important limitation ofthe rank transform strategy.

As a general solution to the group varianceinequality problem, the results of this study do notprovide sufficient evidence to recommend any singleanalysis approach. Before computing hypothesis tests,researchers should first obtain descriptive summarystatistics to determine the sample distributioncharacteristics and to use this information to guidetheir choice of analysis procedures. For mostsituations where the population variances differ, theBrownForsythe procedure can be used to compare means.This procedure has been shown in the present study, aswell as previous investigations, to be generallyrobust to variance inequality and to providestatistical power comparable to or greater thanparametric analysis of variance. There is someevidence however which indicates that when the sampled

78

20

Rank Transform ANOVA

distributions are both skewed and leptokurtic, theBrown-Forsythe procedure can overestimate the nominalsignificance level if sample frequencies and groupvariances are negatively related.

References

Algina, J., & Olejnik, S. F. (1984). Implementing theWelch-James procedure with factorial designs.Educatienal and Psychological keasurement, 44,

39-48.

Brown, M. B., & Forsythe, A. B. (1974a). The smallsample behavior of some statistics which test theequality of several means. Technometrics, 16,

129-132.Brown, M. B., & Forsythe, A. B. (1974b). The ANOVA

and multiple comparisons for data with heterogeneousvariances. Biometrics, 30, 719-724.

Clinch, J. J., & Keselman, H. J. (1982). Parameticalternatives to the analysis of variance. Journalof Educational Statistics, 7, 207-214.

Cohen, J. (1977). StatistiCT1 power analysis for thebehavioral sciences. New York: Academic Press.

Conover, W. J., 41 Iman, R. I. (1981). Ranktransformations as a bridge between parametric andnonparametric statistics. The American Statistician,35 124-129.

Conover, W. J., & Iman, R. C. (1982). Analysis ofcovariance using the rank transformation.Biometrics, 38, 715-724.

Dauphin, M. (1983). Level of significance of therank-transformed t-test in normal populations withunegual variances. Paper presented at theConference of the American Statistical Associaton.

Dijkstra, J. B., & Werter, P. S. P. J. (1981).Testing the equality of several means when thepopulation variances are unequal. CommunicationStatistics - Simulation Computation, B10, 557-569.

79

21

Olejnik and Algina

Fleishman, A. I. (1978). A method for simulatingnon-normal distributions. Psychometrika, 43,521-532.

Glass, G. V., Peckham, P. D., & Sanders, J. R. (1972).Consequences of failure to meet assumptions under-lying the fixed effects analysis of variance andcovariance. Raview of Educational Research, 42,237-288.

Glazer, H. (1963). Comparison of the Student two-sample t-test and the Wilcoxon-Mann-Whitney test fornormal distributions with unequal variances.Dissertation Abstracts, 24/05, 2054.

Iman, R. L. (1974). A power study or 2 rank transformfor the two-way classificatiov mockl when inter-action may be present. The Can.:Ii:n Journal ofStatistics. Section C: Applications, 2 727-239.

Iman, R. L., & Conover, W. J. (1979). The ,st of therank transformation in regression. Technometrics,12, 499-509.

Iman, R. L., Hora, S. C., & Conover, W. J. (1984).Comparison of asymptotically distribution-freeprocedures for the analysis of complete blocks.Journal of the American Statistical Association,79 674-685.

James, G. S. (1951). Tests of linear hypotheses inunivariate ana multivariate analysis when the ra:Aosof the population variances are unknown. Biometrika,21, 19-43.

Johansen, S. (1980). The Welch-James atproximation ofthe distribution of the residual sum, of squares inweighted linear tegression. Biometrika, 67, 85-92.

Kohr, R. L., & Games, P. A. (1974). Robustness of zteanalysis of variance, the Welch procedure and a Boxprocedure to heterogeneous variances. Journal ofExperimental Education, 43, 61-69.

Lee, H., & Fung, K. Y. (1983). Robust procedures formulti-sample location problems with unequal groupvariances. Journal of Statistical ComputationSimulation, 18, 125-143.

Levy, K. (1978). An empirical compariso.1 of the ANOVAF-test with alternatives which are more robustagainst heterogeneity of variance. Journal ofStatistical Simulation, 8, 49-57.

80

22

,

Rank Transform ANOVA

Nath, R., & Duran, B. S. (1981a). A generalization ofthe rank transform in the two-sample locationproblem. Communication Statistical-Theory andMethods, A10, 1437-1455.

Nath, R., & Duran, B. S. (1981b). The rank transformin the two-sample location problem. CommunicationStatistical-Simulation Computation, B10, 383-394.

Ramsey, P. H. (1980). Exact Type I error rates forrobustness of Student's t test with unequalvariance. Journal of Edtriational Statistics, 5,

337-350.

Wang, Y. Y. (1971). Probabilities of the Type I

errors of the Welch tests for the Behrens-Fisherproblem. Journal of the American StatisticalAssociation, 66, 605-608.

Welch, B. L. (1947). The generalization of Student'sproblem when several different population variancesare involved. Biometrika, 34, 28-35.

Welch, B. L. (1951). On en- comparison of severalmean values: an alternative approach. Biometrika,38 330-336.

Wetherill, G. B. (1960). The Wilcoxon test and non-null hypotheses. Journal of the Ro al StatisticalSociety, 23, 402-418.

Yuen, K. K. (1974). The two-sample trimmed t forunequal population variances. Biometrika, 61,

165-170.

AUTHORSSTEPHEN F. OLEJNIK,Associate Professor, Department of

Educational Psychology, University of Georgia,Athens, Georgia 30602

JAMES ALGINA, Professor, Foundations of Education,University of Florida, Gainesville, Florida 32611

2381