Levels Issues in Theory Development, Data Collection, And Analysis

7/29/2019 Levels Issues in Theory Development, Data Collection, And Analysis

1/36

'Aaadmmf ol Maoagem ant Revimr1994. V(d. 19. No. 2. 19S-229.

LEVELS ISSUES IN THEORY DEVELOPMENT, DATACOLLECTION, AND ANALYSISKATHERINE J. KLEINUniversity of MaryloncL College ParkFRED DANSEREAUState University of New York, BuffaloROSALIE J. H A UUniversity of Akron

De


2/36

196 Academy oi Management Review AprilAmericcm histoiy. But, as the conclusions of voting may differ as a func-tion of the level of analysis, so too may the conclusions of research differas a function of the level of analysis. This fact has dogg ed organizationalresearch for decades. It has led to confusion and controversy regardingthe appropriate level of analysis for, and thus the appropriate conclu-sions to be drawn from, research on topics as varied as performanceappraisal, job design, training, pay, leadership, power, participation,communication, climate, culture, technology, organizational perfor-mance, and structure (Campion, 1988; Glide, 1985, 1988; Goodm an, 1987;Klein, R ails, & Douglas, In press; Kozlowski & Kirsch, 1987; Markham,1988; Nachman, Dansereau, & Naughton, 1985; Ostroff & Ford, 1989; Pfef-fer, 1982; Podsakoff & Schriescheim, 1985; Rousseau, 1983, 1985; Yam-marino & Bass, 1991; Yammarino & Naughton, 1988).Entreaties to organizational researchers to address levels issuesmore carefully and exp licitly are not new (e.g ., Behling, 1978; Dansereau,Alutto, & Yammarino, 1984; Glick & Roberts, 1984; Mossholder & Bedeian,1983). Nevertheless, leve ls is su es continue to be a source of ongoing d e-bate w ithin the organizational literature (e.g., G ^ rge, 1990; Yammarino& Markham, 1992).To help resolve the controversy and confusion that surroimds levelsissues, we offer a framework that differs in four key respects from pastwork on levels issues in organizational theory and research. First, ourframework highlights the primacy of theory in addressing levels issues.In contrast, the vast majority of the articles, chapters, and books on leve lsissues in organizational research focus primarily on statistical q[uestions:how to justify aggregation, how to analyze data in accordance with thelevel of a theory, and/or how to analyze multilevel data (e.g., Bedeian,Kermery, & Mossholder, 1989; Bryk & Raudenbush, 1992; Dansereau et al.,1984; Fir eb au gh , 1978; Glick & Rob erts, 1984; Jam es, 1982; Jam es,Demaree, & Wolf, 1984, 1992; Kenny & La Voie, 1985; Kozlowski & Hcrttrup1992; Iincohi & Zeitz, 1980; Mossholder & Bedeian, 1983; Tate, 1985).

One important exception to the emphasis on statistical questions isRousseau's (1985) typology of mixed-level theories. Rousseau's (1985)chapter may, how ever, create the inadvertent impression that attention tolevels is orjy a priority for scholars who undertake mixed-level theory.But, precise articulation of the level of one's constructs is an importantpriority for all organizational scholars whether they propose single- orm ixed-level theories. Further, w e go beyond Rousseau's typologry to clar-ify the profound implications of specifying a given level or levels of atheory. Greater appreciation and recognition of these implications will,we show, enhance the clarity, testability, comprehensiveness, and cre-ativity of organizational theories.


3/36

1994 iTiein, Danseieau, and Hall 197highe r level un its. In contrast, m uch of the existing work on levels issu eswithin organizational theory and research highlights just two of thesealternative s: homogeneity and independ ence (e.g. . Glide, 1985; James etal. , 1984; Kenny & LaVoie, 1985; Kozlowski & Hattrup , 1992). The thi rdalt ern ativ e of heterog ene ity {also known a s a frog-pond or pa rt s effect) isrecognized in some previous organizational work on levels issues (e.g.,Dansereau, Graen, & Haga, 1975; Glick & Rob erts, 1984), bu t its relev an ceand meaning within the organizational literature have been little ex-plored. In discussing the meaning and implications of heterogeneity, weexplicate a fruitful and relatively uncommon avenue for organizationaltheory building.

Third, by clarifying the a ssum ption s that unde rlie the specification oflevels of theory, we illum inate the underlying logic of statistical an aly se sdesigned to test the conformity of data to predicted levels of theory. Ourdiscussion of these issues is deliberately nonquantitative. Instead, weuse a pictorial approach to demonstrate the potential dangers of dataan aly sis an d interpretation when d at a do not conform to the level oftheory. Thus, our treatmen t of these issue s is (a) more broadly acc essib lethan quantitative presentations of levels issues (e.g. , Dansereau et al. ,1984; Glick & Roberts, 1984; Jam es et al. , 1984) and (b) com patible w ith th evariety of statistical indicators that may be used to test levels issues.Finally, we demonstrate the importance of levels issues to topicsspanning the range of organizational science. In contrast, much of thediscussion of levels issues within the organizational literature has oc-curred within the context of a few topic areas, primarily climate (e.g.,Glick, 1985,1988; Jam es, Joyce, & Slocum, 1988) an d leade rship (e.g., Yam-marino & Bass, 1991).We beg in our presen tation w ith a d escription of our terminology a n da brief introduction to levels issues. We then present the framework,successively highlighting levels issues in theory development, data col-lection, and data analysis. We focus initially on single-level theory andresearch, examining multiple-level models in the final section of the ar-

ticle. Each section of the article stresses the importance of theory in re-solving levels issues. Statistical approaches to levels issues have not ridthe field of levels-related ambiguities, controversies, and critic[ues. Atheory-based approach to levels issues may be helpful.

A NOTE ON LEVELS TERMINOLOGYThe organizational literature on levels-of-analysis issues lacks a

widely ac cep ted terminology to describe levels issu es. How, then, shouldautho rs describe levels issues? One app roach is to use very broad term s,


4/36

198 Acadexay ol M anagement B eriew Aprilorgan izational en tities. However, the unfamiliar terminology ma y m ak e itdifficult for rea de rs to gr as p the su bst anc e of the pres entatio n.Accordingly, we have chosen to refer to a specific and familiar orga-nizational contextindividuals within groupsto present our points.Our argument is by no means intended to refer only to work groups,however. Rather, the term group may be interpreted to mean any higherlevel organizational entity, for example, a dyad, team, company, or in-dustry. Similarly, the term individu a/s m ay b e interpreted very broadly torefer to elem ents th at a re nes ted in, or m em bers of, h igher level entities,for exam ple, me mb ers of a dy ad, em ployees within a team , dep artm entswithin a company, or companies within an industry.As noted previously, we focus initially on single-level theories. Ac-cordingly, in the opening sections of the article, we refer to "the level ofa theory." However, some theories ha ve more tha n one level. Further, twotheories m ay use the sam e basic construct or, at least, the sa m e termi-nology(e.g., technology, efficacy) to characterize different organiza-tional en tities. We recognize the se nua nc es , but given the complexity an dab stra ctn es s of levels issu es, we find it helpful to spea k of the level of atbeoxy in the open ing s ection s of this article, shifting late r in the artic le toa more subtle and complex discussion of the level of a construct and thelevels of a tbeoiy.

AN INTRODUCTION TO LEVELS ISSUESBy their very nature, organizations are multilevel. Individuals workin dyads, groups, and teams within organizations that interact with otherorganizations both inside and outside the industry. Accordingly, levelsissues pervade organizational theory and research. No construct is levelfree. Every construct is tied to one or more organizational levels or enti-ties, that is, individuals, dyads, groups, organizations, industries, mar-kets, an d s o on. To exam ine organizational phenom ena is thus to encoun-

ter levels issues.Levels iss ue s create partic ular problem s when the level of theory, th elevel of measurement, and/or the level of statistical analysis are incon-gnient. The level of tbeoiy describes the target (e.g., individual, group,organization) that a theo rist or rese arch er aim s to depict an d ex plain . It is"the level to which g enera lizations ar e m ad e" (Rousseau, 1985: 4). Thelev el of measuiement des cribe s the actu al source of the data "the imit towhich data are directly attached (e.g., self-report data are generally in-dividual level, the nuniber of group members is measured at the grouplevel)" (Rousseau, 1985: 4). The level of statistical analysis describes thetreatment of the data during statistical procedures. For example, if the


5/36

1994 Klein, Vansezeau, cmd Hall 199identical, the obtained results may reflect the level of measurement orstatistic al a na ly sis rathe r than th e level of theory. Moreover, the obta inedresxilts may seriously misrepresent the relationships a researcher wouldhave found if he or she had analyzed the data at the same level as thetheory. In attributing the results to the level of the theory, a researchermay draw an erroneous conclusion or, in the language of levels, commita fallacy of the wrong level (Galtung, 1967; Glick, 1988; Glick & Roberts,1984; H an ey , 1980; Jam es et al ., 1988; Kenny & La Voie, 1985; Mossholder &B ed eian , 1983; Pfeffer, 1982; Robinson, 1950; R ou sse au , 1985).

LEVELS ISSU ES IN THEORY DEVELOPMEN T: THE LEVEL O F THEORYA critical first st ep in add ress ing levels issu es is the specification ofth e level of one's theory. Rousseau, for exam ple, w rote that "theories m ustbe built with explicit description of the levels to which generalization isap pr op ria te " (1985:6). The imp lications of specifying a lev el of theory m aynot be well understood, however. In the following section, we show thatin specifying a level of theory, one implicitly or explicitly predicts thatmembers of a group are homogeneous, independent, or heterogeneouswith respect to the constructs of the theory. Fiulher, in specifying a levelof theory, one predicts (again, implicitly or explicitly) that the relation-ships among the theoretical constructs are a consequence of differencesbetween groups, differences between members independent of groups, or

differences within groups (Dansereau et al., 1984).Specifying Homogeneity: The Group as a Whole

In specifying that the level of a theory is a group, a theorist predictstha t gro up m em bers a re sufficiently similar with respec t to the constructin question th at they may b e characterized a s a whole. He or sh e need notrefer to group m em bers at al l, but only to the group as a who le; a singleva lue or chara cteristic is sufficient to describe th e group . Objective grou psize is perhaps an extreme example; it is clearly invariant across themembers of a group.Hom ogeneity am ong th e mem bers of a group is commonly c onsidereda pre req uisite for asse rting th at the construct in fact appl ies to that grou p(Dan sereau et al. , 1984). Summ arizing this view, Roussea u e xpla ined that"homo geneity w ithin groups on X provides evidence of th e meaningful-n e ss of X at th e un it lev el" (1985: 6). Jam es su gg est ed tha t "the u se ofaggregates [to describe envirormients in psychological terms] is predi-cated on-demonstrat ing perceptual agreement because agreement im-pl ie s a sh ar ed asBigriunent of psycho logical m ea nin g" (1982: 228). M oss-holder and Bedeian commented that "if individual satisfaction is to beag gre ga ted to represen t group mo rale across groups being studied, the re


6/36

200 Academ y of Maixtgemen t Review Aprilto a theoretical construct, a theorist predicts that group members' valueson a given construct are identical. Further, in proposing relationships, heor she focuses on variation between groups, positing that differencesbetween groups on one construct of the theory are related to differencesbetween groups on other constructs of the theory (Dansereau et al., 1984).Examples of the specification of homogeneity abound in the organi-zational literature. They include Ashford and Tsui's (1991) research onfeedback seeking among managers (in which each manager 's subordi-nates are conceptualized and studied as a homogeneous group), Klein's(1987) researc h on em ployee ow nership (in which ea ch com pany's employ-ees are conceptualized and studied as a homogeneous unit) , and Micheland Hambrick's (1992) and Wiersema and Bantel's (1992) studies of top-ma nagem ent team s ( in which dem ographic and other team character is-tics are conceptualized and studied as homogeneous within teams). ^ Ineach of these studies, the target unit is conceptualized as a single, wholeimit and described by a s ing le value .The studies address diverse organizational entities, but they areunited by their assertion of homogeneity within each of their target enti-t ies. We comment on this to underscore the point that numerous organi-zational entities or levels may be conceptualized as homogeneous.^ Toanticipate a later point, we should also note that many of the studiesabove are effectively cross-level studies; they predict that higher levelorganizational properties (e.g., the characteristics of a company's em-ployee owne rship plan) influence, and render hom ogeneous, lower levelorganizational properties (e.g. , employee attitudes toward their com-pany).Specifying Independence: Individuals Free of Group Influence

If the theorist specifies that the level of a theory is the independentindivid ual, he or she p redicts tha t, with respect to the constructs of inter-est, individual members of a group are independent of that group's influ-ence. Thus, the value of a construct for an individual member of a groupis inde pe nd en t of the valu e of the construct for other me m bers of the sa m egro up . Be cau se grroup m em bers hip is irrelevant to the theory's constructs,the distinction of within-group and between-group variation is also

' Michel and Hambrick (1992), WierBama and Bantel (1992), and other organizationaldemographers (e.g.. Jackson et al., 1991; Tsui & O'Reilly, 1989; Wagner, PfeHer, & O'Reilly,1984) used a measure oi within-group variance (the coefficient of variation) to describe theextent of demographic variability within groups. Because this measure of variability is usedto describe a property of ea ch gToup as a whole, that is. the extent to which each group, as


7/36

1994 Kleia, Dansereau. and Hall 201irrelevant (Dansereau et al., 1984). Variation in the constructs is concep-tualized simply as between-individual variation (e.g., the product of in-dividual differences). In conceptualizing the relationship between twoconstructs at this level, one thus suggests that between-individual vari-ability in a construct is related to between-individual variability in an-other construct.Examples of this level of theory also abound in the organizationalliterature. Recent examples include Chen and Spector's (1991) study ofem ployee negative affectivity (independent of job or organization effects,for example); Whitely, Dougherty, and Dreher's (1991) research on man-agers' early career progress (independent of job or organization effects);Gregersen and Black's (1992) study of employee commitment to a parentcompany and a foreign operation (independent of company effects); andGovindarajan and Fisher's (1990) study of strategic business unit perfor-mance (independent of firms and industries).

In the first three examples, the focal entity is the individual em-ployee. In the fourth example, the focal entity is the strategic businessimit. The exam ples are united, however, by their underlying prediction ofindependence. We include Govindarajan and Fisher's (1990) study to un-derscore the point that, as above, numerous organizational entitiesnotjust individualsmay be conceptualized as independent.Specifying Heterogeneity: Individuals Within Groups

The level of som e theories is neither the individual, nor the group, butthe individual within the group. Relatively uncommon within the organi-zational literature, theories of this sort focus on individual attributes rel-ative to the group average for this attribute. These theories describe frog-pond effects (Firebaugh, 1980), also known as within-group effects (Glick& Roberts, 1M4) or parts effects Pansereau et al., 1984). The term frogpond perhaps best captures the comparative or relative effect that is cen-tral to theories of this type: depending upon the s ize of the pond, the verysam e frog may be small (if the pond is large) or large (if the pond is sm all).Theories of this type have two distinctive features, intimated above.First, they predict that the effects of an independent variable (X) on adependent variable (Y) are context dependent. More specifically, "indi-viduals' scores on Y . . . depend not only on their scores on X, but also onthe size of the X relative to those of others in the soc ial group" (Firebaugh,1980: 46). Thus, the sam e X value may y ield different Y values in differentcontexts.' Within one context, a given X value may be relcrtively large.Within a second context, the sam e X value may be lelatively small. Rel-ative, not absolute, value is predictive.


8/36

202 Academy ol MaoagemeDt Review Apiildictor. Not everyone is low. With respect to the constructs of interest,individuals vary within the group.In assertin g that the lev el of a theory is the individual w ithin thegroup, the theorist thus implicitly or explicitly asserts that group mem-bers are neither hom ogeneous nor independen t of the group, but hetero-geneous. Although group members are assumed to vary with respect tothe theory's construct, the group is deemed a meaningful entity. Knowl-ed ge of the group context is not only informative but necessary to interpretan individual's placement or standing in the group. Accordingly, users oftheories of this type posit that within-group variability on one construct(i.e., each individual's standing on a construct relative to the group av-erage for that construct) is related to within-group variability on anotherconstruct.To clarify this description, we outline three exemplars of heterogene-ity from the organizational literature. Each one su gg es ts that the effects ofthe central construct are context dependent. According to vertical dyadiclinkage (VDL) theory, the first exemplar, leaders divide subordinates intoan in-group and an out-group based in part upon the subordinates' rela-tive performance (e.g., Dansereau et al., 1975; Graen & Cashman, 1975).Thus, in the context of one group of subordinates, a n em ployee m ay be arelatively high performer and an in-group member. In the context of adifferent group of subordinates, the same employee, performing at thesame absolute level, may be a relatively poor performer and, thus, anout-group member. Performance relative to the group meannot abso-lute performancedetermines in-group and out-group status.Rafaeli and Sutton's (1991) work on emotional contrast strategiesamong criminal interrogators and bill collectors provides an intriguingqualitative example of heterogeneity. Analyzing "good copbad cop"strategies, Rafaeli and Sutton (1991) suggested that a cop's effectivenessdepends not upon his or her absolute and independent level of "good-ness" or "badness," but upon the contrast effecthow good or bad thecop a p p e a r relative to his or her partner. Indeed, a cop acting the sameway may b e the go od cop in one setting (relative to his or her worse coppartner) and the bad cop in another setting (relative to a different, bettercop partner).A final exam ple com es from the strategy literature (e .g.. Porter, 1980,1985). This literature highlights the context dependence of firm perfor-mance. Indeed, by definition, a firm's competitive advantage is a functionof its competence relative to its competitors. Accordingly, a single firmmay be a superior performer (relative to its competitors) in one market andan inferior performer (relative to its competitors) in a second market.


9/36

1994 Klein. Ban sereau . and Hali 203to its competitors in the beer industry, Phillip Morris is superior at mar-keting. Relative to Coke and Pepsi, Phillip Morris is not.Thus, numerous organizational entities may be conceptualized asheterogeneous. We should also note that theories of this type are oftencategorized a s cross-level theories (e.g., see Ro usseau, 1985) because thefocus on such theories is not just the individual, but the individual incontext: the individual w ithin the group.An Example: Participation and PerformanceHomogeneous,Independent, or Heterogeneous?

An exa m ple m ay clarify the distinction am ong the three a ssum ptionsof variability. This example also illustrates how a given construct may,de pe nd ing u pon th e natu re of the specific theory in question, b e concep-tualized as homogeneous, independent, or heterogeneous.Co nsid er the relations hip of participation a nd performan ce. Both pa r-ticipation and performance may be conceptualized as homogeneouswithin grou ps. A theorist may, for examp le, define pa rticipation a s th eextent to which decision making is shared among the members of thegrou p (a glo ba l characterization of the group) an d ex am ine its relation tothe performance of a group as a whole.Alternatively, both participation and performomce may be conceptu-alized as indep)endent of the group. For exam ple, a theorist ma y posit th atthe propensity to participate in decision making is an individual-levelcharacteristic, as is individual performance, independent of group influ-ence. (Perhaps both participation and performance are a function of in-dividual assertiveness or intelligence.) Group membership is thus irrel-evant to the relationship of participation and performance; individualdifferences determine participation and performance.Finally, participation and performance may be conceptualized asheterog eneo us; the be tter the individual's performance com pared to his orher peers in the gfroup, the more confident he or she may feel to partici-pate in group decision making. In this case, the same individual mayparticipate a great deal in one group (because he or she is a high per-former in this group), but he or sh e may pa rticipa te very little in a secondgroup (because he or she is a low performer in this group). Thus, partic-ipation is a function of individual performance relative to average groupperformance.

As we shift from homogeneity to independence to heterogeneity, themeaning of the constructs and of their relationship to one anotherch an ge s: Although each conceptualization is plausible, ea ch on e evokesa different ex planatory m echanism, sug gests a different strategy for datacollection and analysis, and anticipates a different intervention strategy.(See Y amm arino & Bass, 1991, for a n illustration of this point with reference


10/36

204 Academy a/Management Review Aprilregarding the meaning of the theory and conclusions to be drawn fromtests of the theory.Before describing the implications of the previous discussion for or-ganizational theory building, we examine the applicability of the threeassu m ptio ns of variab ility to diverse topics an d sub discip lines within th eorganizational literature. In addition, we consider a recent theoreticalmodel that, despite its creativity and thoughtfulness, illustrates the am-biguity that m ay ar ise a s a result of insufficient attention to leve ls issu es .Homogeneity. Independence, and Heterogeneity Across OrganizationalEntities and Organizational Subdisciplines

Predictions of homogeneity, independence, or heterogeneity are notlimited to theo ries an d s tud ies of individua ls an d grou ps, but, in fact, theyunderlie the specification of levels throughout organizational theory andresearch . Thus, for exam ple, com panies may b e conceptualized a s homo-geneous within industries, independent of industries, or heterogeneouswithin industries.A second ex amp le observations of individuals over time is moreun us ua l, th us offering an illustration of the flexibility a nd gene ralizabil-ity of the homogeneity-independence-heterogeneity framework (see alsoDansereau et al., 1984). Observations of each individual may be concep>-tualized a s hom oge neo us. In this case, observa tions of a given in divid ualare identical, or stable, over time, but observations of different individ-uals vary significantly on the observed characteristica classic person-ality or dispo sitio na l effect (Staw, Bell, & C lau sen , 1986). Alternatively ,observations may be conceptualized as independent. Here, observationsof each individual are, in fact, independent of the individualthe prod-uct, for example, of situational influences. Finally, observations may beconceptualized as heterogeneous. In this case, observations of each in-dividual over time vary predictably about the mean for each individual.Physical a ctivity over time provides a n exam ple; for every period of h ighphysical activity (running), an individual is likely to have a correspond-ing period of low physical activity (resting).^ In Table 1, we summarizeand list examples of the application of the homogeneity-independence-heterogeneity framework to diverse organizational entities.As the three assumptions of variability cross organizational entities,so they cross the parent disciplines of organizational inquiry. For

^ The concept of ind ividual heterogen eity across time (or heterogeneity within a sing leindividual) is similar but not identical to ipsative concepts and measiues. An iijsative


11/36

1994 Klein, Danseieau. and HallTABLE 1AssumptionB of Variability Across Organizational Entities

205

EntlttmIndividuals

ovei time

Individualswithin groups

Groups withinorganizations

Organizationswithinindustries

HomogeneityObservations of

each individualare homogeneousover time (e.g..dispositionaleffect)

Group members arehomogeneouswithin eachgroup (e.g., stageof groupdevelopment)

Groups arehomogeneouswithin eachorganization(e.g., groupperformancestandards set bythe organization)

Organizations arehomogeneouswithin eachindustry (e.g..nature oforganization'sproducts)

Assumption* of VariabilityIndependence

Observations of eachindividual areindependent overtime (e.g..situational effect)

Group members areindependent ofgroups (e.g., groupmember perceivedwork-familyconflict)

Groups areindependent oforganizations (e.g..frequency withwhich groupmembers socializeas a group outsideof work)

Organizations areindependent ofindustries (e.g..organizationalprovision offamily-orientedbenefits such asparental leave)

HeterogeneityObservations of each

individual areheterogeneous overtime (e.g.. relativelevel of physicalactivity over time)

Group members areheterogeneouswithin each group(e.g.. relative powerof each individualwithin the group)

Groups areheterogeneouswithin eachorganization (e.g..relative performanceof each sales teamwithin eachorganization)

Organizations areheterogeneouswithin each industry(e.g., relative marketshare of eachorganization withinan industry)

example, within differing theories of organizational behavior, psycholo-gists characterize individuals as (a) indep end ent of organizations, (b) ho-mo geneou s within organizations, and (c) heterogeneous within organiza-t ions. Organizational sociologists typically characterize individuals ashomogeneous within organizations. Beyond this, however, differing theo-ries of organ izational sociology describe organizations a s (a) indepe nde ntof industries, (b) homogeneous within industries, and (c) heterog eneo uswithin industries. Entities (individuals versus organizations)not as-siunptions of variability distinguish the parent disciplines of organiza-tional inquiry.Ambigui ty in the Specification of LeveKs) oi Theory: An Exam ple fromthe Organizational Literature


12/36

206 Academy of Man agement Beview Aprilapparent in Lawless and Price's (1992) recent theoretical analysis of tech-nology champions. Their model is fresh and insightful, but it would, un-fortunately, prove difficult to test.

Building on agency theory. Lawless and Price (1992) examined theroles that technology champions and users play during innovation adop-tion. The term champion is clearly and explicitly defined (Lawless &Price, 1992:342). The term usei isnot.As a result, it is unclear whether theauthors conceptualize users as independent individuals or as the homo-geneous mem bers of technology-adopting groups or organizations. Is themodel desicrned to explain between-individual, between-group, or be-tween-organization differences in technology adoption? Should the modelbe tested within a single organization or across organizations? Lawlessand Price's presentation of the model leaves these cjuestions unanswered.The model's propositions do little to resolve these ambiguities. Insome propositions, the authors appear to assume homogeneity withinorganizations: "Nonmonetary rewards and sanctions, disj:>ensed by orga-nizations and their members can enhance the availability and perfor-mance of champions" (Lawless & Price, 1992: 350). This proposition sug-gests a between-organization comparison of the availability andperformance of champions. In some propositions, the authors appear toassume individual independence: "Champions' and users' preferencesare likely to diverge due to personal orientations and perceived net ben-efits from technology implementation" (Lawless & Price, 1992: 351).Siirely, individuals, not groups and organizations, have "personal orien-tations." Finally, in some propositions, the authors appear to assumeheterogeneity within the champion-user relationship: 'The greater thevariance between champion and user skills," [the more likely championsare to] "impose their preferences on users" (Lawless & Price, 1992: 347).This proposition suggests that the extent to which champions imposetheir preferences on users is a function of neither user skills alone, norchampion skills alone, but their relative disparity.Multilevel models of organizational processes as complex as innova-tion adoption are both plausible and apt, but they require (even more thansingle-level theories) explicit and thorough explications of the authors'assum ptions of variability, a point to which we return at the conclusion ofthis article. Had Lawless and Price devoted greater attention to levelsissues, their rich model would have been still more impressive and moretestable.Implications for Theory Development


13/36

1994 Kle in, Oanserecru, and Hall 207Aeterogeaejfy. This specificity increases tbe c larity ofoiganizcrtional theories.

The leve l of a theory sha pes the fundamental nature of the constructsof the theory, determining whether they are to be view ed as descriptors ofhomogeneous groups, independent individuals , or heterogeneousgroups. Further, the level of theory establishes the expected nature of therelationships among the constructs of the theory: Is between-group, be-tween-individual, or within-group variability in one construct of the the-ory predicted to be associated with between-group, between-individual,or within-group variability in a second construct of the theory? These arethe direct and inevitable consequences of specifying a level of theorywhether or not these consequences are explicitly highlighted within thetheory. Thus, we encourage authors to specify explicitly the leve l or levelsof their theories.

2. Theory building may be enhanced by specificationand discus sion of the sources of the predicted homoge-neity, independence, oi heterogeneity o f the cons truc ts.Attention to these issues increases the depth and com-prehens iveness of organizational theories.Given the importance of the leve l of a theory, theorists should includein their specification of the level(s) of their theory an explication of the

underlying assumptions of variability. In short, why are group membersexpected to be hom ogeneous, independent, or heterogeneous? Answeringthis question yields a more comprehensive and convincing theory.The organizational literature provides a rich source of concepts tojustify and explain predictions of homogeneity, independence, or hetero-geneity. Authors predicting homogeneity might, for example, allude to avariety of organizational practices and processes expected to engenderhom ogeneity within groups, including attraction and selection processes(similar ind ividuals are attracted to and selected into a group), socializa-tion (group members receive similar training and indoctrination and,thus, come to respond in a similar fashion), social information processing(individuals are subject to common social influences or commitment pro-cesses), and common experience (group members experience the samegroup events) (e.g ., Pfeffer, 1977; Salancik & Pfeffer, 1978; Schneider, 1987;Schneider & Reichers, 1983; Thom as & Griffin, 1989). Authors predictingindependence might allude to individuals' distinctive personal charac-teristics, individuals' diverse experiences at work, or individuals' lack ofinteraction with other group members (e.g., Staw et al., 1986). Finally,authors predicting heterogeneity might allude to formal ranking proce-dures within a group (e.g., a within-group performance ranking system),organizational processes that require supervisors to differentiate among


14/36

208 Academy of Managem ent Renew April3. Theory building may be enhanced by ex plicit consid-eiation of alternative assu mptions of vaziability. Ex-plicit consideration of alternative assu mptions of vari-ability may increase tbe creativity of organizationaltheories.

Although many organizational constructs can only be conceptualizedas homogeneous or as independent or as heterogeneous, otherslikeparticipation and performancecan be meaningffully conceptualized inea ch of the se w ay s. In such cas es , theorists may find that it refines theirthinking and spurs their creativity to speculate about alternative concep-tualizations of their constructs. In asking, for example, "When is thisconstruct not independent but homogeneous?" a theorist may gain newinsight into the assumptions that underlie his or her theory. Further, he orshe may generate new propositions to be pursued in future theory build-ing and research. Ostroff's (1992) research on the satisfaction-performance relationship at the organizational, rather than individual,level of an aly sis exem plifies the potential benefits of exam ining familiarconstructs at less familiar levels of theory. So, too, does Barley andKnight's (1992) theoretical ana lysis of the preva lence of stress com plaintsamong, not individuals, but specific societal time frames, subcultures,occupations, and roles. Speculation regarding alternative assum ptions ofvariability may provide the initial spark for the development of multiple-level theories, a topic to which we return at the conclusion of the article.

4. In clarifying a nd explicating the level or levels oftheir theories, organizational scholars may discover anew synergy among the diverse subtopics of the field.The previous discussion suggests that assumptions of homogeneity,independence, and heterogeneity span the parent disciplines of organi-

zational theory and research. Thus, despite the sharp differences thatsepara te them (cf. Cappelli & Sherer, 1991; House & Rousseau, 1992; Pfef-fer, 1982), the varying traditions and perspectives of organizational theoryand research share common underlying elements of logic and structure.This commonality suggests that even theorists who differ in focus andorientation may have something to leam from one another if they makecommon assumptions of variability. For example, a theory that predictsthat organizations are homogeneous within industries (e.g., Aldrich, 1979)may be of interest and use to a theorist who predicts that individuals arehomogeneous within organizations (e.g., Schneider, 1987). In this way,greater attention to levels issues and assumptions of variability may


15/36

1994 Klein, Danseteau, and Hall 209LEVELS ISSU ES IN DATA COLLECTION: THE LEVEL OF MEASUREMENT

When the level of a theory is precisely specified and explicated, re-searchers testing the theory may collect data in a way that ensures theconformity of the d at a to the level of theory (e.g., they ma y collect glob al,group-level data that varies only between groups to test a theory speci-fying within-group homogeneity). This is an optimal strategy when thelevel of a theory is certain, w e sug ges t. When the level of a theory is op ento question, however, researchers may find it advantageous to use alter-na tive data-collection strateg ies that allow them to test, rather than sim-ply en su re , th e fit of the d at a to the level of theory. We explore the se leve l-of-measurem ent issu es in the following sections.Data-Collection Strategies Ensuring Conformity to Predictionsof Variability

Re search ers w ishing to test theories that predict within-group homo-gene ity ar e commonly advised (a) to use research m easu res that (like thetheory) focus on the \init as a whole and (b) to maximize between-groupvariability in the research sample (Glick, 1985; Rousseau, 1985). Thus, totest a homogeneous-group theory, a researcher might collect data fromnumerous groups, using an objective measure or expert ratings to obtaina single score representing each group as a whole. In this way, the ho-mogeneity of the data within groups is ensured.R esea rche rs wishin g to test theories that predict individual ind epen-den ce from gro ups a re a dvise d (a) to use m easure s that (like the theory)draw attention to each individual's unique experiences and characteris-tics and (b) to maximize between-individual variability. Thus, for exam-ple, a researcher might employ survey measures, focusing on each indi-vidual's unique experiences and characteristics, in a sample of diverseand independent individuals. The independence of the data from groupsis thus fostered.

Finally, res earc her s w ishing to test theories that predict w ithin-groupheterogeneity are advised (a) to use measures that (like the theory) high-light the position of each individual relative to the group mean and (b) tomaxim ize within-group variability. Thus, for examp le, a rese arche r m ightcollect da ta acro ss a niimber of groups asking a n expert on each g roup tous e a forced choice s cale to rank order the mem bers of each g roup w ithrespect to the construct(s) of interest (e.g., leadership, power, perfor-mance). In this way, the heterogeneity of the data within groups is en-sured.To Ensure Conformity or Test Predictions?


16/36

210 Aca dem y of Management Aeviewr Aprilused a global measure to characterize a group, he or she lacks the dataneeded to tes t whether members are , indeed, homogeneous wi th ingroups on the variables of interest. If a researcher has used a surveyinstrument to collect data from individuals within a single group, he orshe carmot test empirically whether group members' scores are, indeed,independent of group membership.In ca se s in which the level of the theory is beyond q uestion, en surin gthe conformity of data to the level of the theory in this fashion appearscompletely approp riate. Surely a researcher ne ed not ask the mem bersofea ch g rou p how la rg e their group is if the construct of interest is objectivegrou p size. Fiirther, some constructs may ha ve no meaningful an alo gu eat a lower level of theory. For examp le, a grou p's stag e of developm ent orin ter na l p roc ess (e.g., G ersick, 1988, 1989, 1991) m ay on ly be con cep tual-ized and assessed as a homogeneous property of each group.In som e ca se s, however, the level of organizational constructs is opento debate. Organizational climate (Glick, 1985), participation (Klein,Ra ils, & Do ugla s, In press), l ea de rsh ip (Glick & Rob erts, 1984), affect(George, 1990), and technology (Rousseau, 1983, 1985), for example, maybe conceptualized a nd m easured at more than one level. In such cas es, afailure to test empirically the assumptions of the theory regarding thehom ogene ity, inde pen den ce, or heterogeneity of the constructs m ay le av ethe conclusions of the research open to question and criticism.We hast en to add , however, that no data-collection strategy is ' le ve lne utra l." For exam ple, survey and observational mea sures pre sup po se alevel of theory insofar a s they nece ssarily direct respon den ts' attentio n to(a) group homogeneity (e.g., "In general, how do the members of yoiugrou p feel ab ou t X?"); (b) ind ivid ua ls ind ep end ent of grou ps (e.g., "Howsatis fied a re you p erso na lly with X?"); or (c) grou p hetero gen eity (e.g.,"Co m pared to the other mem bers of your group, how great is yoiu X?") (cf.Schneider, 1990).

Sampling strategies also presuppose a level of theory insofar as they(a) allo w or prohibit testing a theory's predictions of hom ogeneity, inde -pe nd en ce , or heterog eneity an d (b) maximize or minimize within-group orbetween-group variability. If a researcher collects survey data from asingle organization, for exam ple, this precludes asse ssm ent of betw een-organization differences in survey responses. Less obviously, data col-lection across organizations within a single industry may minimizeobserved between-organization differences if organizations are homoge-neo us within ind ustries with respect to the variab les of interest.Implicatioiu for Data Collection


17/36

1994 K lein . Z>ansereoru. an d H aii 211conformity of the data to the level of the theory. If thelevel of a theory is open to question, however, research-ers may enhance the fairness and rigor of their researchby employing data-collection strategies that simulta-neously (a) direct respo ndents' attention to the predictedlevel of theory, (b) maxim ize the variability predicted bythe theory, and (c) allow one to test empirically the the-ory's predictions of homog eneity, independence, or het-erogeneity. In all cases, the u se of mu ltiple, diversemeasures is ideal.

This guideline suggests a resolution of seemingly contradictory adviceto (a) collect da ta in a manner that ensures the congruence of level of theoryand measurement (Rousseau, 1985) and (b) collect data at a level belowthe target level of theorydata that can then be aggregated to the levelof the theory (Burstein, 1980). The ideal approach, however, is to employmultiple and varied m easures of the constructs of a theory. When diversem easures of a construct dem onstrate the variability predicted for the con-struct, researchers' confidence in the level of the construct is enhanced.The previous guideline also draws attention to the importance ofmaximizing the variability predicted by the theory, whatever the re-searcher's data-collection strategy. A researcher needs ample between-individual variability to test a theory that predicts individual indejien-dence; ample between-group variability to test a theory that predictswithin-group homogeneity; and ample within-group variability to test atheory that predicts within-group heterogeneity. Anything less invitesrange restriction.These are fundamental research concerns. They may m ake or breaka study. George's (1990) research on group affect, for example, has beencriticized for failure to evidence significant between-group variance andcovariance (Yammarino & Markham, 1992). Arguably, the focus on statis-tics is misplaced. Contrary to the level of her theory, George's measuresas se ss ed individual affect, not group members' assessm ent of the typicalaffect displayed by group members. Further, her sample came from oneorganization, not a sam ple likely to maximize between-group variability.Had George's data collection strategy matched the guideline above, herdata m ight well h ave shown significant between-group variance and co-variance.In sum, the choice of a data-collection strategy is a judgment callinformed in part by (a) the precision and rigor of the \uiderlying theory'sspecification cmd exp lication of the level of the theory, (b) the researcher'sconfidence and the confidence of others in the field in the underlyingtheory's assertions of the level of theory, and (c) a careful analysis of the


18/36

212 Academy ol Managem ent Review April"cheating." Rather, this procedure fosters the construct-level validity ofone's measures.

LEVELS ISSUES IN DATA ANALYSIS: THE LEVEL O FSTATISTICAL ANALYSISLevels scholars have commonly rurged researchers to align their data-analysis strategies with the level of theory. In this section, we suggestthat researchers may enhance the quality of their research not only byaligning their analyses to the level of theory, but also by examining thefit or conformity of the data to the theory's predictions of homogeneity,independence, or heterogeneity. If data have been collected in such away as to ensure the conformity of the d ata to the level of theory, there isclearly no need for analyses of this kind. Accordingly, in this section, weassum e that data a re, in fact, availab le to test the conformity of the d atato the level of theory. Further, as mentioned previously, we focus onsingle-level theories, examining multiple-level theoretical models follow-ing this section.

Aligning Level of A nalysis and Level oi TheoryAs we have noted, scholars (e.g., Glick & Roberts, 1984; Pfeffer, 1982;Rousseau, 1985) commonly advise researchers to align the level of theirstatistical analy ses with the level of their theory. If the level of the theory

is the homogeneous group, researchers are advised to use global, group-level scores, or individual-level scores aggregated to the group in theiranalyses. If the level of the theory is the independent individual, re-searchers are advised to use unaggregated, individual scores to test thetheory. Finally, if the level of theory is the individual w ithin the group,researchers are advised to use deviation scores (or to control for between-group differences in some other way) to test the theory.Although w e conctir with this advice, we dem onstrate in the follow-ing section that if the level of statistical analysis matches the level oftheory, yet the dcrta do not conform to the predicted level of theory, aresearcher may draw erroneous conclusions from the data. The impor-tance of conformity of the d ata to theories predicting within-group homo-geneity is relatively well known and well understood. We demonstratenext that conformity of data to a theory's predictions of heterogeneity orindependence is equally important.Conformity of Data to the Level of a Theory

To specify a level of theory, we have shown, is to predict first that themembers of a group are homogeneous, independent, or heterogeneouswith respect to the constructs of a theory. Second, it is to predict that the


19/36

1994 Klein. D aaseieau, and Hall 213Figure 1 is d esign ed to clarify the meaning and implications of con-formity of data to the level of theory. The figure presents hypothetical datadepicting the relationship of two measures (X and Y] representing 60 em-ployee s organized into six work groups of 10 employees each . The threerows of the figure depict near-perfect conformity of data to predictions of

homogeneity, independence, and heterogeneity, respectively. Figure 1thus illustrates an ideal. In most cases, real-world data are far messierand far more challenging to interpret, a point to which we will return.Each column of the figure depicts a bivariate scatterplot of the ob-served X and Y va lues. The scatterplots in column 1 depict individualobservations; 60 ungrouped individual data points are shown . The scat-terplots in column 2 depict individual observations arranged aroundgroup means; the mean values of X and Y for each of the six groups areshown, as well as the variability of the individual scores around eachmean. The scatterplots in column 3 depict the six group means only.Finally, the scatterplots in column 4 depict the relationship of the X andY va lue s among the individuals within a s ing le prototypical group. Thus,column 1 represents the individual level, unaggregated correlation; col-umn 2 the extent of variability within and between groups; column 3 thecorrelation of the m ean scores; and column 4 a simplified representationof the correlation of the within-group deviation scores.

Row 1 of the figure d epicts conformity of data to predictions of w ithin-group hom ogeneity. A s predicted by the level of theory, the measures arehom ogen eous within groups, showing greater within-group hom ogeneitythan would be expected by chance (column 2). Further, the relationship ofthe measures reflects the correlation of between-group differences (col-umn 3). Indeed, within a single group, the correlation of individual mea-sures of X and Y is near zero (column 4). If the level of the researcher'stheory is the homogeneous group, analyses of the group mean scores(column 3) provide an appropriate assessment of the relationship of themeasures.

Row 2 illustrates conformity to predictions of individual indepen-den ce from groups. The data lack the within-group hom ogeneity depictedin row 1; the v alu es of X and Y for individual observations are indepen-dent of groups. Individual observations are effectively randomly distrib-uted across groups. The group means vary as would be expected bych an ce (column 2).* Accordingly, the correlation of the group m eans

* The data in row 2 of the figure depict some between-group variability even thoughbetween-group differences are expected to be nonsignificant when data conform to predic-tions of independence. Only if a researcher sampled extraordinarily large groups of inde-pendent individua ls would he or she expect to find no between-group variability. (In such acase, however, within-group variability would not exceed what the researcher would expect


20/36

214 Academy o/ Management Beview ApiU

IIo

I 8I MI>I 5DO

ss >I I sCO


21/36

1994 Klein. Danseteaa, and Hall 215(column 3) approximates (but may misrepresent) the correlation betw eenthe two measures w ithin groups (column 4) and across cfroups (column 1).If the level of the researcher's theory is the independent individual, thedisa ggrega te data (column 1) provide an appropriate asse ssm ent of therelationship of the measures.The data depicted in row 3 conform to predictions of heterogeneitywithin groups. The measures are heterogeneous within groups (column2). As frog ponds differ in size , so the groups show mean differences on thepredictor (depicted along the horizontal access) (column 3). However, thedependen t variable (depicted along the vertical access) varies only withingroups, show ing more heterogeneity than expected by chance . The rela-tionship of the two measures is thus entirely within groups. Group mem-bership m ust be taken into accoimt in order to get a true representation ofthe relationship. Accordingly, the correlation of the measures w ithin anysingle group (column 4) is significant, whereas the correlation of thegroup mean scores i s not (column 3). If the level of the researcher's theoryis the individual within the group, the correlation of the within-groupdeviation scores provides an appropriate assessm ent of the relationshipof the two measures.^ (Other analyses of the individual X and Y scoresthat control for group mean differences on the independent variable a lsomay be appropriate.)

Nonconformity of Data to the Level of TheoryWhen data do not conform to the level of the theory, analysis andinterpretation of the data in accordance with the level of theory inviteserroneous conclusions (Robinson, 1950). If the results are significant, aresearcher m ay conclude that his or her theory is supported, w hen in factthe data, in their entirety, do not support the predicted level of theory. Wetum again to Figure 1 to illustrate this risk.If the theory postulates homogeneous groups, yet the data do not

^ Our portrait of heterogeneity differs from that sug ges ted by Dc nser eau and hi s col-leagues' (1384) portrait of "parts." In the ideal parts condition described by Dansereau et al.,Jboth X and Y show minimal between-group variability. In contrast, w e sug gest that in aheterogeneous data set, only one measure, not both, varies so lely within groups. The dif-ference betwee n the two portraits may reflect differing assumptions about the nature of themeasures in a heterogeneou s data set. C onsider an organization in w hich bonus pay isawarded to each employee a s a function oi his or her performance relative to his or her groupmean. In th is case, if performance is measured on an absolute scale, th e observed relation-ship w ill match that dep icted in row 3 of the figure. If the performance m ea stu e in stead


22/36

216 Academy o! Managem ent Review Aprilconform to this level of theory, an observed relationship between them ea n scores may prove m isleading (see Ostroff, 1993). If th e da ta conformto the inde pen den t in divid ual level of theory, the correlation of th e m ea nscores (row 2, column 3) m ay b e significant, yet attribu ting the r esu lts tobetween-group differences is inapprop riate; the da ta provide no eviden cethat the measiues capture meaningful, homogeneous group-level char-acteristics.If the level of theory is the indiv idua l ind epe nde nt of group s, yet thedata do not conform to this level of theory, examining the disaggregatecorrelation alone also may prove misleading. If the data conform to pre-dictions of homogeneity, the disaggregate correlation (row 1, column 1)may be significant, althou gh th e da ta provide no evidence for the a sser-tion that the measures assess individual characteristics, independent ofgrou ps. Similarly, if the d at a conform to prediction s of heterogen eity, thedis ag gre ga te co rrelation (row 3, column 1) ma y be significant, altho ug hthe data do not in fact show individual independence of groups.^Finally, if th e level of one's theory is the ind ividual within th e group ,yet the data do not conform to this level of theory, examining the corre-lation of the deviation scores alone may prove misleading. If the dataconform to predictions of independence, the correlation of the deviationscores (row 2, column 4) may b e significant, although the da ta provide noevidence of systematic heterogeneity within groups.

In sum, aligning the level of one's data analyses with the level oftheory is insufficient to prevent levels fallacies; conformity of data to thelevel of theory is critical. When data do not conform to the level of thetheory, data analyses that are periormed at the predicted level of theoryyield artifactual results . These results may well support the substantivehyp othe ses of the resea rch (e.g., that X pred icts Y). Nevertheless, suchresults rest upon a house of cards blown asunder by closer examinationof the data.Testing the Conformity of Data to the Level of Theory:Statistical Indicators

Statistical tests of the predicted homogeneity, independence, or het-erogeneity of a measure fall into two rough categories: (a) those thatassess the extent of agreement within a single group and (b) those thatassess the extent of agreement (or reliability) by contrasting within- andbetw een-gro up va rian ce . Jame s, Dem aree, a nd Wolf's (1984) r, ^ typifies


23/36

1994 JCiein, D an ser ea u, an d Hall 217the first category. The second category includes analysis-of-variancemodels and their related measures of association such as eta-squared(D ansereau et al., 1984) an d different forms of th e in tr ad a ss correlation(GUck, 1985; Kenny & La Voie, 1985; MaxweU, Camp, & Arvey, 1981). Al-though they employ differing criteria, these indicators are designed toan sw er a common que stion: Does the variability w ithin groups differ iromwhat would be expected by chance?A num ber of statistical indicators also are a vaila ble to test w hetherthe relationship between measures is a function of homogeneity, inde-pe nd en ce , or hetero gen eity (see, for exa m ple, Boyd & Iversen , 1979;Dansereau et al., 1984). Using Dansereau and colleagues' (1984) proce-dures, for exam ple, a researcher can e xam ine the relative m agn itude of(a) the co rrelation of X and Y ba sed upon raw score, disagg rega te d ata; (b)the c orrelation of the g roup m ean s for X an d Y (i.e., the ra w s core da taaggregated to the group level); and/or (c) the correlation of the within-group deviation scores for X and Y.Numerous questions rem ain regarding the various statistical indica-tors l isted previously. What are the strenc^s and weaknesses of thevarious tests? How are the tests related? How do the size of the sample,the sampling strategy, and the characteristics of the data-collection in-strument (e.g., the wording of questionnaire items) affect the results ofthese tests? How should a researcher proceed when, as is often the case,the re sults of the tes ts provide m argin al, not strong, support for the pre-dicted level of theory? Further, how should he or she proceed if the testsyield different con clusions regard ing th e conformity of da ta to the level oftheory? (See, for exam ple , G eorge , 1990; Yam marino & Markham, 1992).Resolution of the qu estion s pose d above is clearly very imp ortant forthe future of organizational research. Such a resolution requires furtherresearch and m athem atical ana lysis beyond the scope of this ar ti d e (see,howeve r. H all, 1988; Kozlowski & Hattrup, 1992; M arkham , 1988; Ostroff,1993; Yammarino & M arkham, 1992). Pending such rese arch a n d an aly sis ,w e can only enco ura ge rese arch ers (a) to recognize the fundam ental pre-dictions of homogeneity, independence, and heterogeneity that the indi-cators o utlined abo ve a re d esign ed to test and (b) to us e m ultiple indica-tors of homogeneity, heterogeneity, a nd indep end ence in their own w ork.Implications for Data Analysis

6. When tbe level of a theory is uncertain and data caeavailable to test tbe conformity of tbe data to the pie-dicted level of tbeory, testing the conformity of tbe datato tbe level of theory enhance s tbe clarity of tbe resultsand reduces tbe risk of a levels fallacy.


24/36

218 Academ y ol Man agement Beview Aprilinterpretation of their results uncertain: Are the results meaningful orspurious? Ostroff's (1992) gen erally very imp ressive study df the relation-ship of aggrregctte teacher satisfaction and school performance illustra testhis risk. Ostroff (1992) tested the within-school homogeneity of her pre-dictor variables (e.g., teacher satisfaction) usin g two formulas for intra-d a s s correlations. All but two of the dependent m easu res of school per-formance were global measures, varying exclusively between schools.She naturally did not (indeed, could not) test the w ithin-school hom oge-neity of these m easu res. Nor, however, did Ostroff test the within-schoolhom ogeneity of the two performance measures that were not globa l mea-sinres: student sa tisfaction and teacher turnover intentions. Ostroff aggre-gate d the se two m easures to the school leve l to as se ss their relationshipwith teacher satisfairtion. The school-level relationship of teacher satis-faction and teacher turnover intentions was by far the strongest of theobserved satisfaction-performance relationships. These results raised thepossibility, noted by Ostroff (1992: 969), that the resu lts were spuriousdue to aggregation inflation and response-response bias. Had Ostroffperformed te sts of the predicted hom ogeneity of all of her agg regate m ea-sures, much of this ambiguity might have been resolved. (See, however,Ostroff's 0993] discu ssion of comparing correlations based on individual-level and aggregated data.)As Ostroff's (1992) study suggests, the above guideline is generally,although not i>erfectly, upheld with regard to tests of predicted homoge-neity prior to aggregation. Figiire 1 and our previous discussion suggestthat tests of predicted independence and heterogeneity are equally im-portant. A dditional research i s needed to explore the likelihood and con-sequences of accepting and rejecting predictions of independence and ofheterogeneity.

7. Tes ting the conform ity of the data to the level of the-ory should build upon, not su bstitute for, precise speci-fication and explication of the level of theory and care-ful development of data-collection procedures.

The data a na lys es w e h ave described are design ed to test a theory'spredictions of homogeneity, independence, or heterogeneity. In the ab-sen ce of sound theory (in which the level of theory is clearly specified andexplicated) and data collection based upon such theory, however, theresults of these analyses are extremely difficult to interpret. Suppose, forexam ple, that a researcher's data conform to the independent individuallevel of theory. Should the researcher conclude that this is the leve l of thephenomenon of interest? Perhaps the results instead reflect characteris-tics of the measurem ent strategy or the samp le. U nless research is clearly


25/36

1994 Klein. Da nsereau. and Hali 219Dean and Snell's (1991) ambitious study of the relationship of inte-grated m anu factur ing a nd job de sig n il lustrates the importance of clearspecification of the level(s) of theory for data interpretation. In the theo-retical introduction to their study. Dean and Snell suggested that inte-

grated manufacturing practices within a plant are most l ikely to be eff i-cacious if job characterist ics within the plant are high in complexity,variety, and interdependence. Further, they posited that the relationshipbetween integrated manufacturing practices and job characterist ics ismoderated by source of organizat ional inert ia (e .g . , p lant s ize) . Thestudy's implied level of theory is thus the plant.Data analyses were, however, conducted at the unit level , not theplant level. That is . Dean and Snell (1991) assessed the effects of plant-level integrated manufacturing upon job characterist ics within three

units (operations, production control, and quality) in each plant. Prior tothese analyses, the authors tested the conformity of the manufacturingpractices to the plant level and the conformity of the job characteristicsdata to the unit level us ing the procedures of James and his co l leagues(1984). Thus, their data an al ys es were statist ically appropriate. H owever,Dean and Snell provided no theoretical rationale for their shift from theplant to the unit level of theory. Nor did they provide a theoretical justi-fication for their assumption of job characteristics' homogeneity withinplants (as implied in the theoretical introduction to the study) or withinunits within a plant (as operationalized in the data analyses) . In theabsence of a theoretical model predicting homogeneity of job character-ist ics w ithin u nits and pred icting between-un it (within-plant) differencesin job characterist ics in response to the same plant-level manufacturingpr actic es. D ea n an d Sn ell's (1991) com plex rese arc h resvilts are difficult tointerpret . In short , statist ical tests and analyses should build upon, notsubstitute, for theory.

MULTIPLE-LEVEL THEORY AND RESEARCHIn 1985, Rousseau issued an eloquent call for greater attention tomult iple- level theory and researchtheoret ica l models and empirica lstudies that are used to s imultaneously examine two or more organiza-t ional entit ies (e.g . , groups and organizations). Interest in this topic hasgrown through the publication of Bryk and Raudenbush's (1992) book de-tail in g ne w statist ical procediures for the ana lys is of m ultiple- level da ta.In the following sections, we explore four multiple- level models: (a)cro ss- lev el .m od els , (b) m ixed-ef fects m ode ls , (c) m ixed-determ inants

m od els , an d (d) m ult i leve l mo de ls . ' In eac h ca se , w e h igh l ight the


26/36

220 Academy of Management Review Aprilassu m ptio ns of variab ility that \mderlie each m odel. The key point in th issection is that careful attention to levels issues facilitates, rather thanprohibits, the formulation and testing of such models. Multiple-leveltheory and research often require scholars to span at least two subdisci-pli ne s (e.g., strat egy a n d micro-organ izational b ehavior). Attention to lev-els issues may help scholars recognize and address both common anddiffering assumptions within the subdisciplines.Cross-level Models

Cross-level theories describe "the relationship between independentan d d ep en de nt va ria bl es a t different levels" (Rousseau , 1985: 20). O rga -nizational cross-level theories most commonly describe the impact ofgroup or organizational factors on individual behavior and attitudes.If a theory predicts that individu al group mem bers respond to a char-acteristic of the group in a comparable or homogeneous fashion, thiscross-level theory predicts, in our lexicon, within-group homogeneity.That is, the theory predicts that both the group characteristic (the inde-pendent variable) and individual behavior (the dependent variable) arehomogeneous within groups. Our previous recommendations for theorydevelopment, data collection, and data analysis are completely applica-ble to this kind of theory.Some cross-level theories ins tea d predict, implicitly or explicitly, th atindividual group members respond to a group-level characteristic in adisparate, rather than homogeneous, fashion. Here, the theory's indepen-dent variab le is hom ogeneous w ithin groups, but the depende nt variab leis not; it varies both within and between groups. However, both concep-tually an d statistically, a homog eneous group-level characteristic canno tpredict within-group variance. The predictive power of a homogeneousgroup-level characteristic is necessarily limited to the percentage of be-tween-gToup variance in the depen dent m easure.Further si>ecification and explication of the level of the theory mayovercome limitations of this kind. Particularly helpful may be carefulidentification of the source of variability in group members' responses tothe hom ogen eous g roup cha racteristic. A theorist might, for exam ple,posit that an independent or heterogeneous characteristic of group mem-bers moderates the relationship of the group characteristic to individualbeha vior. Group m em bers for whom the m oderator is high respon d to thegroup characteristic in one way, whereas group members for whom the


27/36

1994 Klein, Dansereau. an d Hall 221(1988) research in which employee values are hypothesized to moderatethe effects that em ployee ownership ha s on employee attitudes; Oldh am ,Kulik, a n d Ste pin a's (1991) rese arch in which em ployee sk ills an d jo bcha racte ristics a re hypothesized to m ode rate the effects of env ironm entalcharacteristics on employee reactions; and Chatman's (1991) person-organization fit research in which employee values are, effectively, hy-pothesized to moderate the effects that organizational values have onem ployees ' at t i tudes and behaviors.Studies of this kind require precise specification of their moderatedcross-level hypotheses. Such precision is rare, we have found, in purelytheo retical mu ltilevel mo dels. Con sider W oodman, Sawyer, and Griffin's(1993) interactionist, multilevel theoretical model of organizational cre-ativity. O ne of the ir key propositions (Woodman et a l., 1993: 310) st at esthat:

The creative performcmce of individuals in a complex socialsetting is a function of salient individual characteristics, so-cial influences that enhance or constrain individual creativity(e.g., gTOup norms), and contextual influences that enhance orconstrain individual creativity (e.g., organizational rewardstructure).Do indiv idu al cha racteristic s thus mo derate the im pact of social an d con-textual forces? This proposition and others designed to amplify it areambiguous. Greater specificity in addressing levels issues would haveau gm en ted the clarity an d testab ility of this complex, integrcrtive m odel.Despite the difficulty of articulating precise cross-level moderatedtheoretical m odels, da ta collection and analy ses to test thes e models a rerelatively straightforward. Here, the level of each construct (rather thanthe leve l of the theory a s a whole) guid es d ata collection and an aly sis . Forexam ple, m easurem ent of the independent variab le is designed to ens ureor test its homogeneity, measurement of the dependent variable is de-signed to en su re or test its independen ce, an d m easurem ent of the mod-erator is designed to ensure or test its independence or heterogeneity(depending upon the level of the construct).Mixed-Effects ModeU

Mixed-effects theories suggest that a single organizational interven-tion may h av e effects at m ultiple levels of the organization. For exa m ple,an organization's shift from a piece rate to a group-based incentive sys-tem m ay en gen der cha ng es in (a) the image of the organization to outsideobservers, (b) the dynamics of intergroup cooperation within the organi-


28/36

222 Academ y of Management Beview Aprilrepresents a homogeneous level of theory (i.e., homogeneous images ofthe organization among observers of each organization). The second als orepresents a homogeneous level of theory (i.e., homogeneous group dy-namics within organizations characterized by homogeneous incentiveplans). The third hypothesis represerits a cross-level, moderated level oftheory (i.e., the effects of an incentive plan on individual satisfaction varyas a function of individual characteristics). Explicit recognition of thesepredictions may enhance further development and testing of such amixed-level theory.Mixed-Determinants Models

Mixed-determinants models suggest that predictors at a variety oflevels may influence a criterion of interest. For example, market charac-teristics (e.g., the availability of jobs), group characteristics (e.g., diver-sity within the group), and individual characteristics (e.g., job satisfac-tion) have all been hypothesized to influence turnover (e.g., Hulin,Roznowski, & Hachiya, 1985). As mentioned previously, careful specifica-tion and explication of the level(s) of such a theory may clarify how bestto develop and test such theory. We explore three possibilities.First, specification and explication of the level of the theory maysuggest that the relationship am ong the measures is primarily a functionof individual perceptions; perhaps an individual's decision to leave anorganization is a function of his or her perceptions of (a) the current jobmarket, (b) the extent to wh ich he or she fits into his or her work cproups,and (c) his or her job characteristics. In such a case, the theory is perhapsbest conceptualized and tested as an independent, individual-level the-ory. Second, specification and explication of the level of the theory maysuggest that, in fact, a m oderated cross-level theory (as described above)best captures the phenom enon of interest. Perhaps, for exam ple, individ-ual turnover decisions are influenced by the interaction of a homoge-neous organizational characteristic (e.g., the organization's policy for pro-motion from within) and a heterogeneous individual within the groupcharacteristic (e.g., relative expertise). In this case, as above, the inter-action term, like the dependent variable, varies both within and betw eengroups.Finally, a researcher may instead seek to examine the additive ef-fects on individual turnover of (a) a hom ogeneous organization-level char-acteristic (e.g., organizational culture), (b) a heterogeneous characteristicof the individual within the organization (e.g., an individual's relative


29/36

1994 Klein. D anseieau. and H all 223is heterogeneous within organizations cannot predict variance betweenorganizations.The complexity of such research is compounded, however, insofar a sthe predictors may be intercorrelated (perhaps as a result of commonmeasurem ent error), making identification of "pure" hom ogen eous, het-erogeneous, or independent effects impossible.^ Interpretation of suchan alys es is thus difficult, but may b e aided through (a) the careful sp ec-ification of the levels of the theory's constructs, (b) the use of multiplemeasures of the theory's constructs, and (c) the use of multiple s tatisticalindicators to test the predicted variability of the m easures.MultUevel ModeU

Multilevel models "specify patterns of relationships replicated acrosslevels of analysis" (Rousseau. 1985: 22). Here, the relationship betweenthe independent and dependent variables is generalizable across orga-nizational entities. For example, Staw, Sandelands. and Dutton (1981)hypothesized that individuals, groups, and organizations each may re-spond to threats with rigid, overleam ed behaviors that are often inappro-priate for the current situation. Within this theory, individuals, groups,and organizations each are conceptualized as homogeneous and inde-pendent of higher level units. Each of the three basic hypotheses stemsfrom the same multilevel theory and, of course, each may be true. Multi-level theories are thus uniquely powerful and parsimonious. An analysisof the assum ptions of variability underlying each hypothesis (and targetentity) clarifies the theory and suggests ideal data-collection and analy-sis strategies for each hypothesis.Implications for Multiple-Level Theory and Research

The previous discussion sug gests the following guideline.8. When the assumptions of variahiJity are comparablefor both the independent and dependent variables, thatis , when both the independent and dependent variablesare conceptualized to vary solely between groups (ho-mogeneity), or soJeiy within groups fheterogeneityj, orioth within and between groups (independence or aninteraction effect), the precision and rigor of m ultiple-level theories, and tests of such theories, are enhanced.

The previous discussion highligh ts the fundamental observation thatboth theoretically and statistically, constructs or variables that are


30/36

224 Academy of Management Review Aprilhomogeneous within groups can predict only between-group variabilityand constructs or variables that are heterogeneous within groups canpredict only within-group variability. This observation suggests that theexplan atory pow er of a theory is enha nced wh en the assiunptio ns of vari-ability underlying both the independent and dependent variables arecomparable .This, a s we hav e shown, by no m ean s obv iates the development an dtesting of multiple-level theories. Cross-level theories easily conform tothis guideline; both homogeneous cross-level models and cross-levelmodels that specify moderators readily meet this standard, as we haveshown. So, too, do mixed-effects models and multilevel models. Poten-tially more problematic are mixed-determinants models, although inmany cases these, too, may conform to the guideline we suggest.Given the complexity and subtlety of the issues inherent in multiple-level theory and research, particularly careful theory development, datacollection, an d an aly sis are required during the development an d testingof these models. New statistical procedures facilitate the statistical anal-ysis of multiple-level data, but in no way do they obviate questions oftheory an d d a ta collection. Bryk an d R aud enb ush 's (1992) statistic al pro-cedu res may , for exam ple, be use d to as se ss th e correlates of the slopesand intercepts of regressions performed within organizations. The num-ber s are relatively eas y to obtain; making s ens e of them is hard er. Theory,in which levels assumptions are precisely specified, is critical.^

SUMMARY AND CONCLUSIONThe level of a theory defines the properties of a theory's constructsan d th e relation ship b etw een the constructs: Are the theory's constructs tobe conceptualized as homogeneous, independent, or heterogeneous? Is

the relationship of the constructs presumed to be a function of between-group differences, between-individual differences, or within-group differ-ences? These questions underlie and unite theories across the entities,topics, and subdisciplines of organizational inquiry. The failure to an-swer these questions renders theories imprecise, open to misinterpreta-tion, and ripe for controversy.Too often, levels issues are considered the domain of statisticians.W e ha ve tried to show that they are not; first an d foremost, leve ls i ss ue sare the domain of theorists. Greater attention to levels issues will in-crease the clarity, testability, comprehensiveness, and creativity of orga-


31/36

1994 Klein. Dansezeau. and Hall 225during data collection: ensure the fit of the data to the level of theory ortest the fit of the data to the level of theory. The certainty of the level oftheory should, we sug gest, determ ine which strategy a research er shou ldadopt .Researchers are commonly advised to align their data analyses withth e level of theory. This alignm ent alon e will not preven t rese arc he rs frominadvertently drawing unfounded conclusions from their data, however.If the conformity of the data to the level of the theory is not ensured, weargue, it should be tested.The development and testing of multiple-level theories magnifiesth es e conc erns. The s tr e n 0 h of m ultiple-level theories is their complexity;they do not oversimplify organizational realities (Burstein, 1980). Specify-ing the level of each construct within a multiple-level theory aids theo-rists a n d res ea rch ers in manacfing such com plexity. So, too, do efforts toalign the assumptions of variability underlying the independent and de-pendent constructs.Can any part of this discussion illuminate the mysteries of the Amer-ican electoral college to which we alluded in the opening paragraph?Perhaps a little. Theory first, we have argued. What is the theory of theelecto ral co llege? The theory is written in the U.S. Constitution. Preserveand equalize the power of the states, the founding fathers urged. This iswhat the electoral college does. It preserves and equalizesa bitthepower of the sta tes . The electoral college gives the largest state s less, an dthe sm alle st s tat es m ore, influence th an prescribed by the relative size oftheir respective voting popu lations. (States have a s m any electoral votesas they have U.S. Senators and Representatives. Thus, Californiathemost populous sta te ha s 54 electoral college votes to Alask a's theleast popu lous sta te's 3 votes. This 18:1 ratio is much sm aller th an theratio of th e sta tes ' respective voting-age p opu lations: 22 m illion to 378,000or 58:1.) Further, sta tes ar e inviolate; they are recognized an d h onored inthe Constitution. Congressional districts are not. They are subject to po-litical she na nig an s a nd wh ims. Theory first? In the abs enc e of a com pel-ling op posing theory, keep the electoral college and preserve the state-,not Congressional district-, level of theory and analysis.

Would that we were all theoreticians of the caliber of the foundingfathers.REFERENCES

Aldrich. H. E. 1979. Organixaiions and environmeofs. Englewood Cliffs, NI: Prentice Hall.Ashford, S. J., & Tsui, A. S. 1991. Self-regulation for managerial effectiveness: The role of


32/36

226 Academ y o/ Mant rPment BeviewBehling, O. 1976. Som e problems in the philosophy of scien ce o i organizations. Academy ofMaaagement Beview. 3:193-201.Boyd. L. H., & Jversen. G. R. 1979. Contextual ana/ysiK Cooeapts and statistical techniquM.Belmont, CA: W adsworth.Bryk, A. S., &Raudenbush, S. W . 1992. liOeraicfaical linear models. Newbury Park, CA:Sage.Burstein, L. 1980. The an alys is of multilevel data in educational research and evaluation. InD. C. Berliner (Ed.), BeHew olnsetmh in education, vol. 8: 1S8-233. W ashington. DC:American Educational R esearch A ssociation.Campion. M. A. 1988. Interdisciplinary approaches to job design: A constructive replicationwith extensions. Jooxaal o/Applied l^cyciiology. 73: 467-481.Cappelli. P., & Sherer. P. D . 1991. The missing role of context in OB: The need for a m eso-

leve l approach. In L. L. Cummings & B. M. Staw (Eds.), Besearch in oiganisationali>eliaTJor, vol. 13: 55-110. Greenwich. CT: IAI Press.Chatman. I. A. 1991. Matching peo ple and organizations: Selection and socialization inpublic accounting firms. Administratire Sdeoce Quarterly, 36: 459-484.Chen, P. Y.. & Spector, P. E. 1991. Negative afiectivity as the underlying cause of correla-tions between stressors and strain. Jountal ol Applied Psycb ologj. 76: 398-407.Dansereau, F., Alutto, I. A.. & Yammarino, F. I. 1984. Theory testing in oryanizational be-iKJvior; Tbe varient approoch. Englewood Cliffs, NT: Prentice Hall.Dansereau, F., Graen, G., & Haga, W. I. 1975. A vertical dyad linkage approach to leader-

ship within formal organizations: A longitudinal investigation of the role making pro-cess. Oiganizatjonal Behavior and Human Performance. 13: 46-78.Dean, J. W., Jr., & Snell. S. A. 1991. Integrated manufacturing and job design: Moderatingeffects of organizational inertia. Academy of Management Journal. 34: 776-804.Firebaugh, G. 1978. A rule for infen ing individual-level relationships from aggregate d ata.American Sociological B eview, 43: 557-572.Firebaugh, G. 1980. Groups as contexts and frogponds. In E. H. Roberts & L Burstein (Eds.)./sioe s in aggregation: 43-52 . San Francisco: ]ossey-Bass.Gaining, J. 1967. fheoriea and metiiods of social rMearch. New York: Columbia UniversityPress.George. J. M. 1990. Personality, affect, and behavior in groups. Jonm al ol Applied Pxychol-ogy. 75: 107-116.Gersick, C. I. G. 1988. Time and transition in work teams: Toward a new model of groupdevelopment Academy of Management/oumal, 31: 9-41.Gersick, C. J. G. 1989. Marking time: Predictable transitions in task groups. Academy ofManagement Journal. 32: 274-309.Gersick, C. I. G. 1991. Revolutionary change theories: A multilevel exploration of the punc-tuated equilibrium paradigm. Academy o/Management Beview , 16: 11-36.


33/36

1994 Klein, Danseieau, and Hali 227Stow Ed.), Aeseaidli in crganizaiionai behavior, vol. 9; 121-173. Greenwich, CT: JAIPress.

Govindarajan, V., & Fisher, J. 1990. Strategy, control systems, and resource sharing: Effectson business-unit performance. Academr ol Management Joumah 33:259-285.

Graen, G., 8c Cashman, J. F. 1975. A role making model of leadership in formal organiza-tions: A developmental approach. In J. G. Hunt & X,. L. Larson (Eds.). LeaderaUp bon-Hen. Kent, OH: Kent State University Press.

Gregersen, H. B.. & Black, J. S. 1992. Antecedents to commitment to a parent company anda foreign operation. Academy of Management Journal, 35: 65-90.

Hall, R. J. 1988. A lerel ol analyals approacb to construct validity and reiationsiiip issues inperceived climate and job tatiaiaction measures. Unpublished doctoral dissertation.University of Maryland, College Park.

Haney. W. 1980. Units and levels of analysis in large-scale evaluation. In K. H. Roberts & LBurstein (Eds.). Issues in aggregation: 1-16. San Francisco: Jossey-Bass.

House. R. J., & Rousseau, D. 1992. On the bifurcation of OB or il it ain't meso it ain't OB.Unpublished manuscript. The Wharton School, The University of Pennsylvania, Phila-delphia.

Hiilin, C. L.. Roznowski. M., & Hachiya, T. 1385. Alternative opportimities and withdrawaldecisions: Empirical and theoretical discrepancies and an integration. PsychologicalBulletin. 97: 233-250.

Jackson, S. E., Brett, J. F., Sessa, V. I., Cooper, D. M., Julin, J. A., iPeyronnin, K. 1991. Somedifferences make a difference: Individual dissimilarity and group heterogeneity as cor-relates of recruitment, promotions, and turnover. Journal of Applied Piychoiogy, 76:675-683.

James, L. R. 1982. Aggregation bias in estimates of perceptual agreement. Journal ol Ap-plied Psychology, 67: 219-229.

James. L. R.. Demaree, R. J., & Wolf, G. 1384. Estimating within-gToup interrater reliabilitywith and without response bias. Journal of Applied Psychology. 63: 85-38.

James, L. R., Demaree, R. J., & Wolf, G. 1332. i^ An assessment of within-group interrateragreement. Journal of Applied Psychology. 78: 306-303.

James, L. R., Joyce, W. F.. & Slocum. J. W., Jr. 1388. Comment: Orgaruzations do not cognize.Academy of Management Review. 3: 722-735.Kenny, D. A., & La Voie, L. 1985. Separating individual and group effects. Journal of Per-sonality and Social Psychology, 48: 339-348.

Klein, K. J. 1987. Employee stock ownership and employee attitudes: A test of three models.Journal of Applied Psychology Monograph. 72: 313-332.

Klein. K. J., &HaU, R. J. 1988. Correlates of employee satisfaction with stock ownership: Wholikes an ESOP most? Journal of Applied Psychology. 73: 630-638.

Klein, K. J., Rails, S. R., & Douglas, C. In press. Power and participation in the workplace:Lessons for empowerment theory, research, and practice. In J. Rappaport & E. Seidman(Eds.), Hqfidbook ot community psychology. New York: Plenum.

Ko2low8ki, S. W. J.. & Hattrup, K. 1992. A disagreement about within-group agreement:


34/36

228 Academy of Management Review AprilLincoln, J. R., 8c Zeitz, G. 1980. Organizational properties from aggregate data: Separatingindividua

Documents

Levels Issues in Theory Development, Data Collection, And Analysis