Upload
yorid-ahsan-zia
View
212
Download
0
Embed Size (px)
Citation preview
7/21/2019 Organizational Research Methods 2013 Hamann 67 87
1/21
Feature Topic: Construct Measurement in Strategic Management
Exploring the Dimensions of
Organizational Performance:A Construct Validity Study
P. Maik Hamann1, Frank Schiemann1,2,
Lucia Bellora1, and Thomas W. Guenther1
Abstract
Organizational performance is a fundamental construct in strategic management. Recently,
researchers proposed a framework for organizational performance that includes three dimensions:
accounting returns, growth, and stock market performance. We test the construct validity of
indicators of these dimensions by examining reliability, convergent validity, discriminant validity, and
nomological validity. We conduct a confirmatory factor analysis with 19 analytically derived indi-
cators on a sample of 37,262 firm-years for 4,868 listed U.S. organizations from 1990 to 2010. Our
results provide evidence of four, rather than three, organizational performance dimensions. Stock
market performance and growth are confirmed as separate dimensions, whereas accounting returns
must be decomposed into profitability and liquidity dimensions. Robustness analyses indicatestability of our inferences for three dissimilar industries and for a period of 21 years but reveal that
organizational performance dimensions underlie dynamics during years in which environmental
instability is high. Our study provides an initial contribution to the clarification of the important orga-
nizational performance construct by defining four dimensions and validating indicators for each
dimension. Thus, we provide essential groundwork for the measurement of organizational perfor-
mance in future empirical studies.
Keywords
factor analysis, quantitative research, reliability and validity, measurement models, organizational
performance
Organizational performance (OP) is fundamental to strategic management research. Research in this
field builds on the assumption that strategy influences OP (Lubatkin & Shrieves, 1986). Furthermore,
1Faculty of Business Management and Economics, Technische Universitat Dresden, Dresden, Germany2School of Business, Economics, and Social Science, University of Hamburg, Hamburg, Germany
Supplementary material for this article is available on the journals website at http://orm.sagepub.com/supplemental.
Corresponding Author:
P. Maik Hamann, Technische Universitat Dresden (TU Dresden), Faculty of Business Management and Economics, Chair of
Business Management especially Management Accounting and Control, D-01062 Dresden, Germany.
Email: [email protected]
Organizational Research Methods
16(1) 67-87
The Author(s) 2013
Reprints and permission:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/1094428112470007orm.sagepub.com
by guest on February 26, 2015orm.sagepub.comDownloaded from
http://www.sagepub.com/journalsPermissions.navhttp://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://www.sagepub.com/journalsPermissions.nav7/21/2019 Organizational Research Methods 2013 Hamann 67 87
2/21
OP is the most common concept addressed in empirical studies in this field; for example, 28% of 439
empirical articles reviewed by March and Sutton (1997) and 29% of 722 articles reviewed by Richard,
Devinney, Yip, and Johnson (2009) include OP in their research design.
The OP construct refers to the phenomenon in which some organizations are more successful than
others. Aconstructis a conceptual term that researchers define to describe a real phenomenon and is
unobservable by nature (Edwards & Bagozzi, 2000). Consequently, OP is subject to the problem of
unobservables in strategic management research (Godfrey & Hill, 1995, p. 519). This problem is
best described in reference to the predictive validity framework (PVF). The PVF includes two levels:
the conceptual level and the operational level (Bisbe, Batista-Foguet, & Chenhall, 2007). At the con-
ceptual level, theories explain relationships between constructs through propositions. Subsequently,
these propositions are empirically tested at the operational level, at which researchers apply indica-
tors to measure a construct. Indicators are observed scores or quantified records (Edwards &
Bagozzi, 2000). The link between the two levels (i.e., between constructs and their indicators) is
crucial to advances in theoretical relationships between constructs. Only if this link is rigorously
established can empirical findings at the operational level be used to test theoretical propositionsinvolving unobservables at the conceptual level. This link is established by examining construct
validity. Construct validity reflects the correspondence between a construct and a measure taken
as evidence of the construct (Edwards, 2003, p. 329). Construct validity encompasses four criteria:
reliability, convergent validity, discriminant validity, and nomological validity (Schwab, 2005).
Paradoxically, in the past, a majority of strategic management researchers regarded construct validity
and the measurement of constructs as low-priority topics (Boyd, Gove, & Hitt, 2005). Consequently,
unobservables (e.g., OP) have often been measured by single indicators whose construct validity has
rarely been assessed. From the PVF, it follows that related theoretical inferences from such studies are
seriously undermined (Combs, Crook, & Shook, 2005; Starbuck, 2004; Venkatraman & Grant, 1986).
Because of its importance for strategic management research, a growing number of studies exam-ine the measurement of OP. These studies are shown in Table 1 and encompass two groups: (a) fac-
tor analyses of the dimensionality of OP (Devinney, Yip, & Johnson, 2010; Fryxell & Barton, 1990;
Rowe & Morrow, 1999; Venkatraman & Ramanujam, 1987) and (b) reviews of the OP measurement
practices used in strategic management research (Murphy, Trailer, & Hill, 1996; Richard et al.,
2009; Tosi, Werner, Katz, & Gomez-Mejia, 2000). The first group of studies provides evidence
of the multidimensionality of OP. However, these studies disagree on the number of OP dimensions
and do not systematically examine the construct validity of indicators that measure these dimen-
sions. Reviews of OP measurement practice provide evidence that empirical studies in strategic
management research employ a plethora of different and unrelated indicators (Murphy et al.,
1996); for example, Richard et al. (2009) reviewed 213 studies and identified 207 different OP
indicators. In this review, 49% of the studies measure OP with a single indicator despite the
multidimensional nature of OP, and 52% of the studies employ only cross-sectional data sets. How-
ever, none of the aforementioned studies develop a framework of the dimensions of OP at the
conceptual level or examine the construct validity of OP indicators based on such a framework.
Combs et al. (2005) directly address the first gap in the literature and develop a framework of the
OP dimensions based on a synthesis of prior studies that focus on OP dimensions and a review of OP
measurement practices. They divide OP into three dimensions: accounting returns, stock market
performance, and growth. Subsequently, they test the OP framework by conducting a confirmatory
factor analysis (CFA) based on a correlation matrix of five OP indicators derived from a meta-
analysis. Despite the significant contribution made by Combs et al., their study has three limitations.
First, Combs et al. do not offer clear definitions of the OP dimensions. Specification of the concep-tual domain and clear definitions of constructs are prerequisites for construct validity (Schwab,
2005). Second, the CFA with three factors and five OP indicators does not satisfy the two-
indicator rule of model identification (Kline, 2011). Consequently, Combs et al. offer only
68 Organizational Research Methods 16(1)
by guest on February 26, 2015orm.sagepub.comDownloaded from
http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/7/21/2019 Organizational Research Methods 2013 Hamann 67 87
3/21
Table1
.PreviousStudiesThatExaminetheDim
ensionsofOrganizationalPerformance.
Study
Numberof
Dimensions
Number
of
Indicators
Numberof
Studies/
samplesize
Method
DimensionsofO
rganizationalPerformance
Operational
Performance
Reviewsand
meta-analyticstudies
Combs
,Crook,andShook
(2005)a
3
5
Notreported
(238studies)
Narrativereviewand
meta-analytic
al
CFA
Accountingreturns
Growth
Stockmarket
Operational
performance
Tosi,Werner,Katz,
andGomez-M
ejia(2000)a
8
30
Notreported
(137studies)
Meta-analyticalEFA
Absolutefinancial
performance
Changein
financial
performance
Stock
performance
Internal
performance
indicators
Returnonequitys
hort
term
Marketreturn
Returnonequitylongterm
Richard,D
evinney,Y
ip,a
nd
Johnson(2009)
3
n/a
213
Narrativereview
Financialperformance
Shareholder
return
Productmarket
performance
Murphy,Trailer,
andHill(1996
)b
4
n/a
52
Narrativereview
Efficiency
Size
Liquidity
Profit
Studiesusingpr
imaryorsecondarydata
Devinney,Yip,andJohnson
(2010)
4
10
Notreported
EFA
Accountingmeasure
Salesmeasures
(salesgrowth)
Marketvalue
Cashflow/profitability
dimension
RoweandMorrow(1999)
3
10
311(2
,398
firm-years)
CFA
Financial(accounting)
Stockmarket
Subjective
reputation
rating
Murphyetal.(1
996)b
9(4)
19(8)
995
(586)
PCA (C
FA)
Liquidity
Salesmeasures
(salesgrowth)
Size
Profitability
Profitgrowth
Salesefficiency
Incomeefficiency
Absoluteincome
Employeeefficiency
FryxellandBarto
n(1990)
2
4
168
CFA
Accounting-basedmeasures
Market-based
measures
VenkatramanandRamanujam
(1987)
3
3
86
MTMMandCFA
Profitability
Salesgrowth
Profitgrowth
Note:WeallocatethedimensionsoforganizationalperformancetotheframeworkofCombsetal.(
2005),whichisshowninboldfa
ce.T
hisframeworkalsoseparatesope
rationalperformanceand
organizationalpe
rformance.C
FA
confirmatoryfacto
ranalysis;EFA
exploratoryfactora
nalysis;PCA
principalcomponentsanalysis;MTMM
multitrait-multimethodmatrix.
aCombsetal.(2
005)andTosietal.(
2000)onlyreporttheoverallnumberofprimarystudie
sthattheyuseintheirreviews.Thisn
umberisprovidedinparentheses.
bMurphyetal.(1
996)conductanarrativereviewandan
empiricalanalysisthatisbasedonther
esultsoftheirreview.C
onsequently,w
eincludethisstudyinbothlists.F
urthermore,theresultsoftheir
exploratoryPCA
andtheirCFAaredifferentinsofarastheCFAencompassesonlyasubsetoftheindicatorsthatareemployedintheP
CA
.WepresentdetailspertainingtotheirCFAinparentheses.
69by guest on February 26, 2015orm.sagepub.comDownloaded from
http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/7/21/2019 Organizational Research Methods 2013 Hamann 67 87
4/21
7/21/2019 Organizational Research Methods 2013 Hamann 67 87
5/21
2005). Common performance indicators, such as growth in market share, product quality, patent
filings, or marketing effectiveness, measure distinct dimensions of operational performance.
In contrast, OP is defined as the economic outcomes resulting from the interplay among an orga-
nizations attributes, actions, and environment (Combs et al., 2005, p. 261). The definition of OP
corresponds to measurement practices in strategic management research because a majority of
researchers assess OP based on economic indicators (Murphy et al., 1996; Richard et al., 2009).
Thus, OP is synonymous with the concepts of financial performance or corporate economic perfor-
mance (Fryxell & Barton, 1990). OP is relevant to both research and practice because in the legal
system (i.e., bankruptcy law or commercial law) and in economic theory, OP (i.e., economic out-
comes) constitutes the final aim of economic activities.
Combs et al. (2005) propose a consistent OP framework with three dimensions: accounting
returns, stock market performance, and growth.
Accounting returns are defined as the historical performance of organizations that is assessed
through the use of financial accounting data as published in annual reports (Fryxell & Barton,
1990). As shown in Table 1, Combs et al. (2005) argue for a single accounting returns dimension,whereas other studies identify several dimensions that are derived from accounting returns indica-
tors. However, we expect at least two separate dimensions to be reflected by accounting returns
indicators. First, a liquidity dimension, which is defined as a firms ability to meet its financial obli-
gations based on cash flows generated from its current operations, is expected (Weygandt, Kimmel,
& Kieso, 2010). Second, a profitability dimension, defined as an organizations efficiency in utiliz-
ing production factors to generate earnings, is expected. Accounting research highlights the differ-
ence between earnings (e.g., net profit) and cash flows that is traced to revenue and expense accruals
(e.g., Dechow, 1994). Accruals mitigate timing and matching problems associated with the alloca-
tion of cash flows to single periods but are subject to distortions caused by discretionary accounting
choices (e.g., a depreciation method or the useful life of assets). Additionally, Rappaport (1993)stresses the divergence between the accounting-based return on investment and the cash flow rate
of return.
Stock market performance reflects the perceptions of investors regarding organizations future
performance (Fryxell & Barton, 1990). This dimension is measured using capital market indicators,
such as total shareholder return (TSR). However, capital market indicators are also influenced by the
momentum and volatility of capital markets, the economy, and psychological effects (Richard et al.,
2009). Stock market performance reflects future OP, in contrast with accounting returns, which
entail a historical perspective. As shown in Table 1, previous studies provide consistent evidence
regarding stock market performance as a distinct OP dimension.
Organizational growth is defined as a change in an organizations size over time. Organizational
growth is a dynamic construct that is commonly evaluated based on three concepts of size: sales,
employees, and assets (Weinzimmer, Nystrom, & Freeman, 1998). As shown in Table 1, previous
studies that investigate the OP dimensions focus on sales growth and disregard employment and
asset growth.
Previous examinations of the dimensionality of OP are subject to three limitations. First, the
number of indicators used is small. For example, Fryxell and Barton (1990) use four indicators, and
Venkatraman and Ramanujam (1987) employ three indicators. However, a small number of indica-
tors may not capture the entire conceptual domain of a construct. Second, indicators are often not
chosen analytically. For example, Murphy et al. (1996) chose 19 OP indicators based on their fre-
quent usage by researchers. These indicators include absolute returns (e.g., net income), return ratios
(e.g., return on assets), size (e.g., number of employees), and ratios of balance sheet items (e.g., debtto equity). Given the conceptual domain of OP, the adequacy of some of these indicators is question-
able; for example, size and static balance sheet items differ conceptually from OP (Combs et al.,
2005; Tosi et al., 2000). If indicators are chosen inadequately, spurious factors may emerge or true
Hamann et al. 71
by guest on February 26, 2015orm.sagepub.comDownloaded from
http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/7/21/2019 Organizational Research Methods 2013 Hamann 67 87
6/21
factors may be obscured in factor analyses (Fabrigar, Wegener, MacCallum, & Strahan, 1999).
Third, cash flow return indicators are absent in the majority of previous studies. Devinney et al.
(2010) and Rowe and Morrow (1999), who both include a single cash flow return indicator in their
factor analysis (cash flow return on sales and cash flow return on equity, respectively), are excep-
tions. This limitation is important because we expect the single accounting returns dimension pro-
posed by Combs et al. (2005) to divide into two dimensions (i.e., liquidity and profitability) when the
convergence of cash flow returns and profitability indicators is examined systematically.
Research Design
Assessment of Construct Validity
During the process of construct validation, four criteria are evaluated: reliability, convergent valid-
ity, discriminant validity, and nomological validity (Schwab, 2005). We employ CFA to examine
construct validity. First, the theory-testing approach of CFA is appropriate for the evaluation of thetwo competing models, the three-OPdimension model and the four-OPdimension model, that
emerged from our discussion of previous research. Second, this approach enables an examination
of the overall fit of a measurement model to a data set. Third, CFA permits researchers to test the
significance of factor loadings. Fourth, CFA supplies indices that provide insights into reliability,
convergent validity, and discriminant validity (Bagozzi, Yi, & Phillips, 1991; OLeary-Kelly &
Vokurka, 1998). Table 2 presents the methods and indices that are applied to assess the criteria
of construct validity (see also Bagozzi & Yi, 1988).
Prior to the assessment of the construct validity criteria in a CFA, the overall fit of the measure-
ment model to the data must be established (Anderson & Gerbing, 1988). The assessment of the
overall measurement model fit to the data is based on the chi-square statistic, the Comparative FitIndex (CFI), the root mean square error of approximation (RMSEA), the standardized root mean
square residual (SRMR), and the Akaikes Information Criterion (AIC). The methodological liter-
ature criticizes the use of definite cutoff criteria for these goodness-of-fit indices. Goodness-of-fit
indices are sensitive to the misspecification of a model and to sample size, model types, and data
non-normality. Consequently, definite cutoff criteria may yield a high Type I error (i.e., rejecting
acceptable misspecified models) if they are too conservative (Marsh, Hau, & Wen, 2004). We
account for this cutoff criteria ambiguity by differentiating between cutoff criteria for acceptable and
good fits of the measurement model to the data and by reporting more than one goodness-of-fit
index, as recommended by Hu and Bentler (1999).2 We compare the competing models of OP based
on their overall measurement model fit to the data. Hereafter, we employ the best fitting model to
examine the four criteria of construct validity.
Reliability is defined as the ratio of systematic variance to total variance (i.e., the degree to which
an indicator is free of random error). Reliability is a necessary prerequisite for validity (Schwab,
2005). Convergent validity is defined as the extent to which multiple indicators represent a common
construct. A number of indicators of the same construct should exhibit high levels of covariance to
be considered valid measures of the construct in question (Bagozzi et al., 1991). In contrast, discri-
minant validity is defined as the degree of divergence among indicators that are designed to measure
different constructs (Edwards, 2003). The methods that we apply to assess these criteria of construct
validity are presented in Table 2.
Nomological validity is based on evidence pertaining to the relationships between measures of
the construct under investigation and measures of other constructs. This evidence should be consis-tent with relevant theory or with the results of previous empirical studies (Schwab, 2005). Conse-
quently, we test the relationships between the dimensions of OP and the determinants and
consequences of OP. Capon, Farley, and Hoenig (1990) conducted a meta-analysis of the
72 Organizational Research Methods 16(1)
by guest on February 26, 2015orm.sagepub.comDownloaded from
http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/7/21/2019 Organizational Research Methods 2013 Hamann 67 87
7/21
Table2
.StatisticsandMethodsThatAreAp
pliedtoAssessConstructValidity
.
StepsinAssessing
ConstructV
alidity
AssessmentCriteria
ExplanationandThresholdsforAcceptability
Overallfito
fthe
measurem
ent
modelto
thedata
Chi-squarestatisticofthe
likelihoodratiotest
H0hypothesisoft
helikelihoodratiotestistheexactfitofaspecifiedmodeltoapop
ulation
(MacCallum,B
row
ne,&
Sugawara,1996
,p.1
32).
AcceptanceofH0
:pvalue>.0
5.
ComparativeFitIndex(CFI)
TheCFIdescribestherelativeimprovementinthefitofthemodelincomparisonwiththefitofthe
independencemodel.Thus,thisindexovercomessa
mplesizeeffects(Bentler,1990
,p
p.245-246)
.
Acceptablefit:CF
I>.9
0;goodfit:CFI>.9
5.
Rootmeansquareerrorof
approximation(RM
SEA)
TheRMSEAmeasuresthediscrepancybetweenthecovariancematrixestimatedfromthemodeland
theobservedmatrix.Thiscriterionadjustsforthemodeldegreesoffreedom(MacCallumetal.,
1996
,p.1
34).
Acceptablefit:RM
SEA .40)a
Cash flow return per employee .648***Cash flow return on sales .692***Cash flow return on assets .692***Return per employee .791***Return on sales .687***Return on assets .748***Employment growth .472***Sales growth .440***Assets growth .638***Total shareholder return .960***Sharpe ratio .723***
Jensens alpha .695***Treynor ratio .777***
Reliability of constructsConstruct reliability (> .60) a .863 .896 .761 .937Average variance extracted (> .50)a .677 .742 .517 .789
Discriminant validity: Fornell-Larcker criterionb
Liquidity .677Profitability .563 .742Growth .010 .047 .517
Stock market performance .017 .027 .048 .789Discriminant validity: Chi-square difference testc
Liquidity 3,241.46*** (3) 8,706.33*** (3) 9,386.71*** (3)Profitability 6,953.09*** (3) 11,473.98*** (3)Growth 14,101.30*** (3)
Nomological validity: Antecedent constructsd
Research and development intensity () .227*** .271*** .043*** .012*Capital investment intensity () .097*** .112*** .051*** .006Market concentration () .022** .018* .034*** .012**Market share () .053*** .054*** .014*** .013***
Nomological validity: Consequent constructsSurvival () .022** .040*** .007 .016***
Note:n 37,272 firm-years.aThe thresholds for item reliability, construct reliability, and average variance extracted are given in parentheses.bThe Fornell-Larcker criterion of discriminant validity is satisfied if the average variance extracted for a factor is greater thanits squared correlations with all other factors. The average variance extracted is presented on the diagonal.cThe chi-square difference test is performed between two two-factor models. In the first model, the correlation between thetwo factors is constrained to 1.0. In the second model, this correlation is freely estimated. A significant chi-square differenceindicates discriminant validity. Differences in degrees of freedom are given in parentheses.dNomological validity is tested in the 4FM-B. In this model, all measures of the antecedent constructs are regressed on eachsingle performance dimension, and each single performance dimension is regressed on survival. Comparative Fit Index (.934),root mean square error of approximation (.036), and standardized root mean square residual (.042) indicate the fit of thismodel. The expected signs are given in parentheses. The regression coefficients are presented in boldface if they are statis-tically significant and display the expected sign.*p< .05. **p< .01. ***p< .001.
80 Organizational Research Methods 16(1)
by guest on February 26, 2015orm.sagepub.comDownloaded from
http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/7/21/2019 Organizational Research Methods 2013 Hamann 67 87
15/21
extracted are above the thresholds for all factors, with the lowest values, .761 and .517, respectively,
determined for the growth dimension.
As Table 4 shows, 4FM-B provides evidence of convergent validity for all factors. First, the
factor loadings of all indicators exhibit acceptable convergence (i.e., l> .5). Second, if we consider
the stronger criteria of good convergence (i.e.,l > .7), all indicators of the liquidity, profitability,
and stock market performance factors are above this threshold. The convergence of the three growth
indicators is slightly weaker. The factor-loading estimate of the employment growth indicator (l
.687) and the sales growth indicator (l .663) are both statistically significant but slightly below the
threshold for good convergence.
As shown in Table 5, all factors in model 4FM-B demonstrate discriminant validity. The Fornell-
Larcker criterion holds for all factors. The chi-square difference tests, which compare fixed and
freely estimated two-factor models for all pairs of factors, support this conclusion. As Table 4 shows,
the liquidity and profitability factors exhibit the highest correlation (r .750) among the four fac-
tors. All other correlation coefficients are considerably lower (i.e., r< .25). These results indicate
four dimensions of the OP construct.As shown in Table 5, our analyses provide evidence of nomological validity with regard to the
four OP dimensions. Ten out of 16 regression coefficients of the antecedents of OP are statistically
significant and display the expected signs, according to Capon et al.s (1990) meta-analysis of 320
primary studies. The majority of OP indicators that are included in their meta-analysis belong to the
profitability and liquidity dimensions. Accordingly, all regression coefficients between the four
determinants and these two dimensions are statistically significant and in the expected directions.
Three regression coefficients of survival for the OP dimensions are significantly different from zero
and show the expected (i.e., positive) signs (liquidity, profitability, and stock market performance).
However, growth appears to be unrelated to the survival of companies in our sample.
Robustness Analyses
Table 6 presents the results of our robustness analyses. Our inferences regarding the construct valid-
ity of 4FM-B are stable across industries and time periods. We repeat our primary analyses of the
construct validity for each industry and each year separately. The overall model fit is acceptable for
all three fit indices in all three industries and in 18 out of 21 years.
The model fit is lowest for years with high environmental instability, as indicated by the volatility
and the annual return of the S&P 500 index (e.g., in 2002, after the burst of the dotcom bubble, and in
2008 and 2009, during the financial crisis). In particular, sales growth and employment growth fail
to demonstrate item reliability and convergence for years with high environmental instability. With
regard to the growth dimension, only the asset growth indicator demonstrates acceptable values forreliability and convergence for all years. The average variance extracted for the growth dimension is
below the threshold for almost every year after 2000 except 2005, indicating a weak construct relia-
bility for the last 11 years. The discriminant validity of the dimensions of OP is evident in all indus-
tries. However, for five of the years, the correlation coefficient between the profitability and
liquidity dimensions is high (i.e., r > .8). Consequently, during these years, the two dimensions
do not discriminate as strongly as implied by the primary analysis. Overall, our results generally
remain unchanged across industries and time periods.
Discussion and ConclusionsThe results of this study reveal the existence of four independent OP dimensions: liquidity, profit-
ability, growth, and stock market performance. The evidence of the construct validity of the four-
dimensional OP measurement scheme is strong and consistent across different time periods and
Hamann et al. 81
by guest on February 26, 2015orm.sagepub.comDownloaded from
http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/7/21/2019 Organizational Research Methods 2013 Hamann 67 87
16/21
Table6
.StabilityoftheMeasurementModel4FMAcrossIndustriesandTime.
Fit
Statistics
Reliability
Validity
S&P500
Group
N
CFI
RMSEA
SRM
R
AIC
Chi-Square
Item
R
eliability
Construct
Reliability
Average
Variance
Extracted
Convergent
Validity
Discriminant
Validity
Volatility
Annual
Return
Industries
ICB2000
16,2
43
.943
.044
.04
7
441,800
1,821.91(56)
13/13
4/4
4/4
13/13
12/12
n/a
n/a
ICB5000
10,0
05
.950
.041
.04
4
282,383
979.99(56)
13/13
4/4
3/4
13/13
12/12
n/a
n/a
ICB9000
11,1
04
.950
.048
.04
4
278.172
1,486.66(56)
13/13
4/4
4/4
13/13
12/12
n/a
n/a
Time
1990
513
.950
.050
.06
2
14,9
86
131.53(57)
11/13
4/4
4/4
11/13
12/12
4.9214
3.1
0%
1991
970
.921
.071
.06
0
25,0
23
333.48(56)
13/13
4/4
4/4
13/13
12/12
5.6230
30.4
7%
1992
1,026
.949
.058
.05
1
27,6
79
252.88(57)
13/13
4/4
4/4
13/13
12/12
2.6822
7.62%
1993
1,097
.960
.054
.04
2
29,6
27
232.52(56)
13/13
4/4
4/4
13/13
12/12
2.9739
10.0
8%
1994
1,163
.944
.063
.02
7
21,9
29
310.88(56)
13/13
4/4
4/4
13/13
12/12
2.0597
0.85%
1995
1,601
.941
.064
.06
1
43,7
31
427.74(56)
12/13
4/4
4/4
13/13
12/12
9.2497
37.0
5%
1996
1,820
.940
.062
.04
9
47,8
70
447.95(56)
13/13
4/4
4/4
13/13
12/12
6.2587
22.9
6%
1997
1,976
.949
.055
.05
8
51,8
27
384.76(56)
13/13
4/4
4/4
13/13
12/12
9.2017
33.3
6%
1998
2,125
.945
.058
.05
2
55,8
34
454.01(56)
13/13
4/4
4/4
13/13
12/12
6.4972
28.5
8%
1999
2,124
.951
.055
.04
4
49,1
37
410.97(56)
13/13
4/4
4/4
13/13
12/12
4.6309
21.0
4%
2000
2,419
.907
.063
.07
3
66,6
79
585.18(56)
11/13
4/4
3/4
13/13
12/12
3.8732
8.8
1%
2001
2,402
.905
.067
.07
1
64,2
64
662.28(56)
12/13
4/4
3/4
12/13
12/12
7.0106
11.89%
2002
2,352
.898
.070
.05
9
65,2
62
697.31(56)
11/13
4/4
3/4
13/13
12/12
11.0
146
22.10%
2003
2,243
.915
.056
.03
7
48,4
33
448.34(56)
13/13
4/4
3/4
13/13
10/12
8.6373
28.6
8%
2004
2,158
.949
.054
.04
7
42,0
20
413.94(56)
12/13
4/4
3/4
13/13
10/12
2.9923
10.8
8%
2005
2,042
.929
.055
.05
7
47,6
12
404.52(56)
13/13
4/4
4/4
13/13
10/12
2.8477
4.77%
2006
2,001
.955
.047
.05
6
52,6
09
306.65(56)
13/13
4/4
3/4
13/13
12/12
4.3365
15.2
3%
2007
1,926
.946
.047
.06
9
52,9
65
290.61(56)
10/13
4/4
3/4
13/13
11/12
3.3149
5.49%
2008
1,829
.855
.072
.06
2
54,7
05
581.51(56)
10/13
4/4
3/4
13/13
12/12
15.1
583
37.00%
2009
1,777
.889
.066
.04
9
37,5
70
492.22(56)
11/13
4/4
3/4
13/13
12/12
12.7
387
26.4
6%
2010
1,698
.909
.066
.04
4
43,0
64
473.18(56)
12/13
4/4
3/4
13/13
11/12
5.2141
15.0
6%
Note:Satisfie
dcriteriaarepresentedinboldface.Re
liabilityandvalidityarereportedasthe
proportionofindicatorsorfactorsthatsatisfyeachcriterion.T
hevolatilityof
theS&P500iscalculated
asthenormalizedstandarddeviationbasedonthed
ailyreturnindexineachyear.C
FI
ComparativeFitIndex;RMSEA
rootm
eansquareerrorofapproximation;SR
MR
standardizedroot
meansquare
residual;AIC
AkaikesInformation
Criterion;ICB
IndustryClassificatio
nBenchmark.
82by guest on February 26, 2015orm.sagepub.comDownloaded from
http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/7/21/2019 Organizational Research Methods 2013 Hamann 67 87
17/21
industries. We demonstrate that the accounting return dimension that Combs et al. (2005) propose
should be decomposed into the two distinct dimensions of liquidity and profitability. Organizations
differ in their ability to meet financial obligations based on cash flows generated from their current
operations and in their efficiency to utilize production factors to generate earnings. For years with
high environmental instability, the overall fit of the measurement model to the data becomes weaker.
This result contributes to the findings of Fryxell and Barton (1990), who provided evidence on
changes in the structure of the measurement of OP between years with low and high environmental
instability.
Our study contributes to research on OP measurement in three ways. First, regarding the concep-
tual level of the PVF, we clarify the OP construct. Second, regarding the link between the conceptual
and operational levels of the PVF, we establish the construct validity of 13 OP indicators. Third, we
reveal the dynamics of OP measurement over time.
First, construct clarity encompasses definitions, semantic relationships, contextual conditions,
and coherence (Suddaby, 2010). We offer definitions of OP and its dimensions, which capture essen-
tial characteristics, avoid circularity, and are parsimonious. Additionally, we integrate OP and itsdimensions into a hierarchy of related performance constructs, with organizational effectiveness
at the top of this hierarchy. Furthermore, we provide evidence that the nature of the OP construct
is highly sensitive to environmental instability as an important contextual condition. Researchers
who study OP should be aware of the four different (not interchangeable) dimensions and their con-
textual conditions.
Second, we develop a set of OP indicators at the operational level of the PVF. In addition, we test
the construct validity of these OP indicators based on the four OP dimensions. Thus, we contribute to
the establishment of a link between the conceptual and operational levels of an important construct
in strategic management research. We empirically confirm that hybrid indicators should be avoided
when measuring OP and its dimensions, as recommended by Combs et al. (2005). Additionally, wepropose a measurement scheme of 13 OP indicators that measure all four dimensions in a construct-
valid and parsimonious manner.
Third, environmental instability influences the measurement structure of OP. Researchers should
carefully control for this factor if they address OP in their research design. We recommend that
researchers avoid a nonlongitudinal measurement of OP for years characterized by high environ-
mental instability.
Our findings have important implications for future strategic management research. First, strate-
gic management theories that address variations in OP must consider the four OP dimensions as an
entity or only concentrate on selected dimensions. This implication is further underscored by our
evaluation of antecedents of OP. Profitability and liquidity are influenced by all four tested antece-
dents, whereas our results regarding the growth and stock market performance dimensions present a
different picture of these relationships. This issue must be addressed at the conceptual level, and it
offers fruitful avenues for future theoretical research on the determinants of OP. Second, empirical
researchers addressing OP in their work are encouraged to use the four-dimensional OP measure-
ment scheme, for which construct validity has been established. Our study contributes to the reduc-
tion of the plethora of OP indicators that may be employed in future empirical studies and thus may
facilitate an increase in rigor and relevance in strategic management research.
Our study has three limitations. First, we concentrate on listed organizations because the stock
market performance dimension is applicable to only these organizations. Thus, the question of
whether the construct validity of the other three OP dimensions also holds in organizations that are
not active in the capital market remains unanswered. Second, we employ secondary, objective OPdata according to the recommendation of Dess and Robinson (1984). The question of whether the
four OP dimensions are also applicable to other performance data, such as perceptual OP indicators,
remains unanswered. Third, OP is only one of several important performance constructs (e.g.,
Hamann et al. 83
by guest on February 26, 2015orm.sagepub.comDownloaded from
http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/7/21/2019 Organizational Research Methods 2013 Hamann 67 87
18/21
operational performance or corporate environmental performance). These other performance con-
structs are also subject to measurement issues. Consequently, developing and testing construct-
valid measurement schemes for these constructs offers possibilities for future research.8 Such
research may draw on the methodology of this study and employ the four-dimensional OP measure-
ment scheme to test nomological validity.
Valid measurement is the sine qua non of science. If the measures used in a discipline have not
been demonstrated to have a high degree of validity, that discipline is not a science (Peter, 1979, p.
6). In this respect, our study contributes to the valid measurement of the most important construct in
strategic management research.
Acknowledgments
We would like to thank Mark Orlitzky, Christoph Trumpp, and two anonymous reviewers for their insightful
comments on previous versions of this manuscript. All remaining mistakes are our own.
Declaration of Conflicting InterestsThe author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publi-
cation of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Notes
1. We thank an anonymous reviewer for highlighting the importance of construct clarity.
2. We thank an anonymous reviewer for drawing our attention to this relevant issue.
3. Survival is usually measured by a categorical variable that represents an organizations enduring presence inthe market (Richard, Devinney, Yip, & Johnson, 2009, pp. 732-734). We measure survival as the proportion
of years during which an organization is present in the stock market. We calculate the proportion during each
year of our time period based on the equation s (tey)/(2010 y), withs survival,t
e the year before a
company is delisted or the last year within our time period (i.e., 2010), andy year under consideration.
This operationalization is coarse-grained, but it corresponds with Baker and Kennedys (2002, p. 326) study.
4. We acknowledge that there are alternative measures for assets (e.g., capital employed) and for net profit (e.g.,
EBIT). Thus, ratios that use net profit as the nominator and assets as the denominator are exemplary for an
entire set of potential accounting return indicators. In online supplement 3, we extend our findings to these
indicators.
5. We calculate growth rates in all instances based on the [(tn
tn1)/tn 1] formula (Weinzimmer, Nystrom, &
Freeman, 1998, p. 253).
6. We present an extended sample description including descriptive statistics (e.g., correlations, skewness, kur-
tosis, intraclass correlation coefficients, and design factors) in online supplement 1.
7. Multilevel confirmatory factor analysis (CFA) is another method that accounts for the dependence of our
data. We examine the robustness of our analysis with regard to this methodological decision by conducting
a two-level CFA. Results of this two-level CFA are similar to our primary results and are presented in online
supplement 2. We thank an anonymous reviewer for drawing our attention to this issue.
8. We thank an anonymous reviewer for highlighting this limitation.
References
Akaike, H. (1974). A new look at the statistical model identification.IEEE Transactions on Automatic Control,19(6), 716-723. doi:10.1109/TAC.1974.1100705
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recom-
mended two-step approach.Psychological Bulletin,103(3), 411-423. doi:10.1037/0033-2909.103.3.411
84 Organizational Research Methods 16(1)
by guest on February 26, 2015orm.sagepub.comDownloaded from
http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/7/21/2019 Organizational Research Methods 2013 Hamann 67 87
19/21
Bagozzi, R. P., & Baumgartner, H. (1994). The evaluation of structural equation models and hypothesis testing.
In R. P. Bagozzi (Ed.), Principles of marketing research (pp. 386-422). Cambridge, MA: Blackwell.
Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of
Marketing Science, 16(1), 74-94. doi:10.1177/009207038801600107
Bagozzi, R. P., Yi, Y., & Phillips, L. W. (1991). Assessing construct validity in organizational research.
Administrative Science Quarterly,36(3), 421-458.
Baker, G. P., & Kennedy, R. E. (2002). Survivorship and the economic grim reaper.Journal of Law, Economics,
and Organization, 18(2), 324-361. doi:10.1093/jleo/18.2.324
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin,107(2), 238-246.
doi:10.1037/0033-2909.107.2.238
Bercovitz, J., & Mitchell, W. (2007). When is more better? The impact of business scale and scope on long-term
business survival, while controlling for profitability.Strategic Management Journal,28(1), 61-79. doi:10.10
02/smj.568
Bisbe, J., Batista-Foguet, J.-M., & Chenhall, R. (2007). Defining management accounting constructs: A meth-
odological note on the risks of conceptual misspecification.Accounting, Organizations and Society, 32(7-8),
789-820. doi:10.1016/j.aos.2006.09.010
Boyd, B. K., Gove, S., & Hitt, M. A. (2005). Construct measurement in strategic management research: Illusion
or reality.Strategic Management Journal, 26(3), 239-257. doi:10.1002/smj.444
Burt, R. S. (1976). Interpretational confounding of unobserved variables in structural equation models.
Sociological Methods & Research,5(1), 3-52. doi:10.1177/004912417600500101
Capon, N., Farley, J. U., & Hoenig, S. (1990). Determinants of financial performance: A meta-analysis.
Management Science,36(10), 1143-1159.
Carlson, K. D., & Herdman, A. O. (2010). Understanding the impact of convergent validity on research results.
Organizational Research Methods,15(2), 17-32. doi:10.1177/1094428110392383
Combs, J. G., Crook, T. R., & Shook, C. L. (2005). The dimensionality of organizational performance and itsimplications for strategic management research. In D. J. Ketchen (Ed.),Research methodology in strategy
and management (Vol. 2, pp. 259-286). Amsterdam: Elsevier.
Dechow, P. M. (1994). Accounting earnings and cash flows as measures of firm performance: The role
of accounting accruals. Journal of Accounting and Economics, 18(1), 3-42. doi:10.1016/0165-
4101(94)90016-7
Dess, G. G., & Robinson, R. B., Jr. (1984). Measuring organizational performance in the absence of objective
measures: The case of the privately-held firm and conglomerate business unit. Strategic Management
Journal,5(3), 265-273. doi:10.1002/smj.4250050306
Devinney, T. M., Yip, G. S., & Johnson, G. (2010). Using frontier analysis to evaluate company performance.
British Journal of Management,21(4), 921-938. doi:10.1111/j.1467-8551.2009.00650.xEdwards, J. R. (2003). Construct validation in organizational behavior research. In J. Greenberg (Ed.),
Organizational behavior: The state of the science(2nd ed., pp. 327-371). Mahwah, NJ: Erlbaum.
Edwards, J. R., & Bagozzi, R. P. (2000). On the nature and direction of relationships between constructs and
measures.Psychological Methods,5(2), 155-174. doi:10.1037/1082-989x.5.2.155
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor
analysis in psychological research.Psychological Methods,4(3), 272-299. doi:10.1037/1082-989X.4.3.272
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobserveable variables and
measurement error. Journal of Marketing Research, 18(1), 39-50.
Fryxell, G. E., & Barton, S. L. (1990). Temporal and contextual change in the measurement structure of finan-
cial performance: Implications for strategy research. Journal of Management,16(3), 553-569. doi:10.1177/014920639001600303
Godfrey, P. C., & Hill, C. W. L. (1995). The problem of unobservables in strategic management research.
Strategic Management Journal,16(7), 519-533. doi:10.1002/smj.4250160703
Hamann et al. 85
by guest on February 26, 2015orm.sagepub.comDownloaded from
http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/http://orm.sagepub.com/7/21/2019 Organizational Research Methods 2013 Hamann 67 87
20/21
7/21/2019 Organizational Research Methods 2013 Hamann 67 87
21/21
Venkatraman, N., & Ramanujam, V. (1987). Measurement of business economic performance: An examination
of method convergence. Journal of Management,13(1), 109-122. doi:10.1177/014920638701300109
Weinzimmer, L. G., Nystrom, P. C., & Freeman, S. J. (1998). Measuring organizational growth: Issues, con-
sequences, and guidelines.Journal of Management,24(2), 235-262. doi:10.1177/014920639802400205
Weygandt, J. J., Kimmel, P. D., & Kieso, D. E. (2010). Accounting principles. Hoboken, NJ: Wiley.
Author Biographies
P. Maik Hamann is research assistant and PhD candidate in Management Control/ Strategic Management at
the Technische Universitat Dresden. P. Maik Hamann received his bachelors degree from the University of
Paisley in Scotland and his German diploma degree from the Technische Universitat Dresden in Germany.
He is also lecturer of management accounting at the Technische Universitat Dresden and part-time for Dresden
International University. His main research interests encompass the design of corporate planning systems,
effects of corporate planning at the organizational level, contingency theory, measurement of organizational
effectiveness, construct validity, and philosophy of science.
Frank Schiemann is an Assistant Professor of Accounting at the University of Hamburg, Germany. He
received his PhD degree at the Technische Universitat Dresden, Germany. He was/is lecturer of management
accounting at the Technische Universitat Dresden, University of Hamburg and part-time for Dresden Interna-
tional University. His research focuses on firm valuation models as well as determinants and effects of firms
voluntary and mandatory disclosure decisions via different communication channels. His methodological focus
is on quantitative analysis methods, especially panel data models.
Lucia Bellora is a PhD candidate in Management Accounting and Management Control at the Technische
Universitat Dresden, Germany. She received her German diploma degree from the Technische Universitat in
Dresden. Lucia Bellora is also a lecturer of management accounting at the Technische Universitat Dresden, and
part-time at the Dresden International University and at the International Graduate School Zittau. Her research
interests include the design and performance effects of management control systems, the disclosure of extra-financial information, and the validity of construct measurement. Her methodological interest is directed
especially towards quantitative analysis methods with a focus on structural equation modeling.
Thomas W. Guenther is a professor of Management Accounting and Control at Technische Universitat
Dresden. Thomas Guenther received his PhD and habilitation degree from University of Augsburg, Germany.
He has been a visiting professor several times at the University of Virginia and was/is teaching in MBA and
executive programs at Wirtschaftsuniversitat Wien, Austria; European Business School (EBS), Wiesbaden,
Germany; and Mannheim Business School, Germany. His work covers two fields of research: first, the design
of management control systems within management accounting and strategic management research, and
second, the measurement, valuation, and control of intangibles in financial and management accounting. He
is editor-in-chief of the Journal of Management Control and is on the editorial board of the Business
Administration Review (BARev). He also serves as board member of the Schmalenbach Association. His meth-
odological background is in quantitative analysis, especially structural equation modeling, meta-analyses and
panel data models.
Hamann et al. 87