This article was downloaded by: [York University Libraries]On: 13 August 2014, At: 09:51Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

Structural Equation Modeling: A MultidisciplinaryJournalPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/hsem20

Comparing Squared Multiple Correlation CoefficientsUsing Structural Equation ModelingJoyce L. Y. Kwana & Wai Chanb

a The Hong Kong Institute of Educationb The Chinese University of Hong KongPublished online: 04 Apr 2014.

To cite this article: Joyce L. Y. Kwan & Wai Chan (2014) Comparing Squared Multiple Correlation CoefficientsUsing Structural Equation Modeling, Structural Equation Modeling: A Multidisciplinary Journal, 21:2, 225-238, DOI:10.1080/10705511.2014.882673

To link to this article: http://dx.doi.org/10.1080/10705511.2014.882673

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Structural Equation Modeling: A Multidisciplinary Journal, 21: 225–238, 2014Copyright © Taylor & Francis Group, LLCISSN: 1070-5511 print / 1532-8007 onlineDOI: 10.1080/10705511.2014.882673

Comparing Squared Multiple Correlation CoefficientsUsing Structural Equation Modeling

Joyce L. Y. Kwan1 and Wai Chan2

1The Hong Kong Institute of Education2The Chinese University of Hong Kong

In social science research, a common topic in multiple regression analysis is to compare thesquared multiple correlation coefficients in different populations. Existing methods based onasymptotic theories (Olkin & Finn, 1995) and bootstrapping (Chan, 2009) are available butthese can only handle a 2-group comparison. Another method based on structural equationmodeling (SEM) has been proposed recently. However, this method has three disadvantages.First, it requires the user to explicitly specify the sample R2 as a function in terms of the basicSEM model parameters, which is sometimes troublesome and error prone. Second, it requiresthe specification of nonlinear constraints, which is not available in some popular SEM softwareprograms. Third, it is for a 2-group comparison primarily. In this article, a 2-stage SEM methodis proposed as an alternative. Unlike all other existing methods, the proposed method is simpleto use, and it does not require any specific programming features such as the specification ofnonlinear constraints. More important, the method allows a simultaneous comparison of 3 ormore groups. A real example is given to illustrate the proposed method using EQS, a popularSEM software program.

Keywords: squared multiple correlation coefficients, structural equation modeling, modelreparameterization, multi-sample analysis

In multiple regression analysis, the sample squared multiplecorrelation coefficient, R2, measures the proportion of totalvariance on the dependent variables, Y, that is accounted forby a set of predictors, X1, X2, . . . , Xk. It provides an estimatefor the overall predictive power of a set of predictors. Socialsciences researchers are often interested in comparing R2 indifferent model conditions because this helps them to betterunderstand the relative importance of a given set of predic-tors. One typical comparison is to compare R2 of a givenregression model in different independent samples. Thiscomparison allows researchers to decide whether a givenset of predictors performs equally well in different popula-tions. For example, an educational psychologist might askif parental involvement, teacher’s influence, and personalitytraits together provide the same degree of predictive poweron academic performance of different groups of students(e.g., boys and girls).

Correspondence should be addressed to Wai Chan, Department ofPsychology, The Chinese University of Hong Kong, Shatin, New Territories,Hong Kong. E-mail: wchan@psy.cuhk.edu.hk

Different methods have been proposed for comparingthe squared multiple correlation coefficients in two popu-lations. One common approach is Olkin and Finn’s (1995)asymptotic confidence interval (ACI) for δ = ρ2

1 − ρ22 , the

difference between two population squared multiple correla-tion coefficients. Given R2

1 and R22 are the sample estimates of

ρ21 and ρ2

2 obtained from large independent samples of sizesN1 and N2, respectively, d = R2

1 − R22 is asymptotically dis-

tributed as a normal variate and the 100(1 – α)% ACI for δ is

[d − z(1− α

2 )σ̂ , d + z(1− α

2 )σ̂]

, (1)

where z(1− α2 )

is the 100(1 − α

2

)th percentile point of the

standard normal distribution, and

σ̂ =√

4

N1R2

1(1 − R21)

2 + 4

N2R2

2(1 − R22)

2(2)

is the asymptotic standard error estimate (ASE) of d. Alfand Graf (1999) also developed formulas for the ACI of

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

226 KWAN AND CHAN

δ in different cases. Although they approach the problemsomewhat differently, their derived formula for the ASEof d in independent samples is the same as Equation 2.In addition to the construction of a confidence interval, theASE can be used to obtain a standard hypothesis test ofH0: ρ2

1 = ρ22 (Algina & Keselman, 1999).

Olkin and Finn (1995) proposed a second method fortesting H0: ρ2

1 = ρ22 by using a transformation FZ for R2

1and R2

2 based on Fisher’s Z variance-stabilizing transforma-tion for zero-order correlation coefficients; that is, FZ

(R2) =

loge

(1+

√R2

1−√

R2

). A test statistic for H0: ρ2

1 = ρ22 can be obtained

by using FZ(R2):

Z = FZ(R21) − FZ(R2

2)√4

N1+ 4

N2

. (3)

Given large samples, this statistic is asymptotically dis-tributed as a standard normal variate under H0 (Olkin &Finn, 1995). Nevertheless, it has been argued that usingthe standard normal as the asymptotic sampling distribu-tion of the test statistic in Equation 3 is not appropriate.Gajjar (1967) shows that the limiting distribution of FZ(R2)fails to approach normality as sample size increases. As thesquared multiple correlations only cover the range from 0 to1, the transformed values, FZ(R2) are severely truncated inthe lower tail. Hence, the resulting distribution is positivelyskewed (Alf & Graf, 1999).

Based on Monte Carlo results, Algina and Keselman(1999) showed that Olkin and Finn’s ACI method failsto perform accurately in certain model conditions. This isbecause, first, the ASE in Equation 2 might not be accurateenough when sample sizes are small. Second, the methodrelies on the normal approximation to provide the percentilepoint, z(1− α

2 ), which might not be accurate enough either

(Chan, 2009). Instead, the method might perform better if thepercentile point is empirically estimated from the observeddata rather than using the normal approximation. As a result,Chan (2009) proposed a unified bootstrap procedure for esti-mating the standard error of d and constructing the bootstrappercentile interval (BP) and bootstrap standard interval (BCI)for δ (Efron & Tibshirani, 1993). Simulation results indicatethat the performance of the asymptotic method and boot-strap procedure, in general, are comparable to each otherwith normal data. However, when data are nonnormal, thebootstrap procedure is more robust against nonnormality andit outperforms its asymptotic counterpart (Chan, 2009).

It is obvious that both Olkin and Finn’s (1995) asymptoticmethods and Chan’s (2009) bootstrap method provide stan-dard ways for comparing ρ2 in two populations. However,these methods fail to generalize multiple-group comparisonsbecause both the hypothesis test and the confidence inter-val are based on d, or the function of it, and hence theyare limited to testing the hypothesis, H0: ρ2

1 = ρ22 . As pointed

out by Chan (2009), a challenging research question is “toexplore the possibility of generalizing the existing methodto the case of multiple-group comparison” (p. 583). In fact,there are many situations in which researchers might need totest the equality of ρ2 in three or more groups. For example,in the meta-analytic study using regression, one has to firstdecide whether the effect sizes (R2) from different primarystudies are homogeneous when pooling the data. In otherwords, one has to test the equality of ρ2 across the studiesbefore doing any subsequent analysis. Similarly, in a clin-ical setting, a clinical psychologist might be interested inknowing if the effectiveness of cognitive behavior therapycan be explained by factors such as treatment type, durationand intensity of treatment, and mode of treatment equallywell across patients with different types of anxiety disor-ders. The psychologist thus has to test the equality of ρ2 ingroups of patients with panic disorder, generalized anxietydisorder, obsessive–compulsive disorder, and posttraumaticstress disorder. To conclude, although there is a practicalneed for researchers to make a multiple-group comparison,such a method is still not readily available. The aim of thisstudy is, therefore, to provide a general solution for testingH0: ρ2

1 = ρ22 = . . . = ρ2

m (with m ≥ 2). Specifically, we areto propose a method for comparing squared multiple corre-lations in different groups by using the structural equationmodeling (SEM) technique. Before going into the details ofthe proposed method, we first review the existing methods ofusing SEM as a tool for comparing ρ2 in two groups.

COMPARING ρ2 USING SEM

SEM is a flexible and powerful statistical technique forstudying a variety of models and it is continually gainingpopularity among applied researchers (Hershberger, 2003;Tremblay & Gardner, 1996). Methodologists have studiedhow different models can be formulated in SEM; for exam-ple, regression analysis (Bentler, 1995; Jöreskog & Sörbom,1996), canonical correlation (Fan, 1997), multivariate analy-sis of variance (MANOVA; Cole, Maxwell, Arvey, & Salas,1993), and meta-analytic SEM (Cheung & Chan, 2005), toname a few. In addition, the development of user-friendlyand powerful software programs allows researchers to per-form different SEM analyses with ease. The flexibility ofSEM and advancement of software programs enable SEM tobe a useful tool for comparing squared multiple correlationsin different groups.

Phantom Variable Approach

Cheung (2009) proposed a general method for constructingthe confidence intervals for a number of statistics and psy-chometric indexes including R2 with the use of phantomvariables under the SEM framework. A phantom variableis a latent variable without observed indicators and has no

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

SQUARED MULTIPLE CORRELATION COEFFICIENTS 227

residual. It usually has no substantive meaning but is createdto parameterize constraints on the model (Rindskopf, 1984).Many SEM programs nowadays have functions for creatingnew parameters, which help to simplify the model spec-ification involving phantom variables (e.g., AP Keywordin LISREL [Jöreskog & Sörbom, 1996] and NEW optionin Mplus [Muthén & Muthén, 2007]). By defining a newparameter as the difference of R2 between two groups, userscan readily obtain the parameter estimate, its standard error(SE), and the Wald statistic (cf. Cheung, 2009). Hence, statis-tical inferences about the equality of ρ2 between two groupscan be made.

Although the phantom variable approach provides amethod for comparing ρ2 in two groups in SEM, this methodhas several limitations. First, the success of the implementa-tion of the method depends entirely on one’s ability to writeout the functional expression of R2. The method requires theuser to explicitly specify the new parameter; that is, the dif-ference of R2 between two groups, as a function in termsof the basic SEM model parameters. This procedure canbe troublesome and error prone. Applied researchers mighthave difficulties implementing the method because they haveto fully understand the functional relationship among themodel parameters before they are able to specify the newparameter correctly. Second, the method requires the specifi-cation of nonlinear constraints because the new parameter isdefined as a nonlinear function of the basic model param-eters. Consequently, it cannot be used with some popularSEM software programs such as EQS (Bentler, 1995) andAMOS (Arbuckle, 2007), which do not support the specifica-tion of nonlinear constraints. Third, the method is primarilyfor a two-group comparison. In fact, this approach is basi-cally equivalent to the statistical test using the ASE of din Olkin and Finn’s (1995) ACI method. In addition, thephantom variable approach is the same as Chan’s (2009)bootstrap method if one constructs the BCI or BP for the newparameter, d. In other words, like other existing methods, thephantom variable approach also fails to deal with the com-parison of ρ2 in three or more groups. It might be possibleto extend the phantom variable method to the comparisonof three or more groups simultaneously. Nevertheless, thisdepends critically on the SEM program’s capacity to providean overall test for the new parameter concerned. As far as weknow, most SEM programs fail to perform such an overalltest (Kwan & Chan, 2011). Therefore, applied researchersstill lack a method of making a multiple-group comparisonof ρ2.

In this article, we use the flexibility of SEM on modelspecification and propose a general method for comparingthe squared multiple correlations across different groups.Unlike the existing SEM methods, which depend criticallyon the special features (e.g., specification of nonlinear con-straint) supported by a specific class of SEM softwareprograms, the implementation of the proposed method doesnot require any advanced programming features but the

basic functions available in almost all SEM programs on themarket. More important, the proposed method is readily ableto conduct multiple-group comparisons, whereas the cur-rently existing methods fail to do so. The proposed methodis introduced in the next section. A real example that illus-trates the implementation of the proposed method using EQS(Bentler, 1995) is given. In addition, a simulation study isconducted to compare the performance of different methodsfor comparing ρ2 in different groups.

THE PROPOSED METHOD

The proposed method compares the squared multiple corre-lation coefficients by using model reparameterization, whichis the process in which the hypothesized model is trans-formed into a set of successive covariance-equivalent modelsthat share the same implied covariance matrix as the originalmodel so that a coefficient that does not exist as a modelparameter in the original model becomes a model parameterin the final transformed model (Chan, 2007). Chan’s (2007)sequential model-fitting method for comparing the indirecteffects and Kwan and Chan’s (2011) two-stage approachfor comparing standardized coefficients in SEM demonstratesome of the applications of the model reparameterizationtechnique. In this article, we further extend the usage of themodel reparameterization technique on the study of squaredmultiple correlation coefficients.

Similar to the method proposed by Kwan and Chan(2011), the proposed method is a two-stage approach forcomparing ρ2 across groups. At Stage 1, the original model(M1) of each group is transformed into the R2 model (M2) sothat the squared multiple correlation coefficient, which doesnot appear as a free model parameter in the original model,becomes one of the model parameters in the transformedmodel. Then at Stage 2, the ρ2 in different groups is com-pared by imposing linear between-group constraint(s) onthe parameters of interest in M2 and statistical inferenceis performed by using the likelihood ratio (LR) test. Thefollowing section gives a detailed description of the modeltransformation at Stage 1.

General Framework of Model Transformation atStage 1

We first define the model of theoretical interest. Suppose wewant to fit a regression model with k predictors, X1, X2, . . . ,Xk in m groups as in the model shown in Figure 1a. We labelthis model as the original model, M1. Let γi (i = 1, 2, . . .,k) be the unstandardized regression coefficient of the pre-dictor, Xi; �ij be the covariance between Xi and Xj; and Ebe the error term. Without loss of generality, all variables areassumed to have zero means. Let us consider the model equa-tion of the original model for group p (p = 1, 2, . . . , m) in astandardized form; that is,

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

228 KWAN AND CHAN

(a) (b)

(c)

FIGURE 1 A general regression model for group m with k predictors on Y. (a) Original model (M1). (b) Half-transformed model. (c) Final R2 model (M2).

g(θ ) =k∑

i=1φii + 2

∑∑i �=j

(ai )(aj )φij in panel c. Parameters in parentheses are fixed parameters.

Y =k∑

i=1

γiXi + E

Y

SD(Y)=

k∑i=1

(SD(Xi)

SD(Y)γi × Xi

SD(Xi)

)+ E

SD(Y)

(4)

Because var(

YSD(Y)

)= var

(Xi

SD(Xi)

)= 1, taking variance

on both sides of Equation 4 gives us

1 =k∑

i=1

γ ∗i

2 + 2∑∑

i�=j

γ ∗i γ ∗

j �∗ij + var(D) (5)

where γ ∗i = SD(Xi)

SD(Y) γi; �∗ij = corr(Xi, Xj) = �ij

SD(Xi)SD(Xj); and

D = ESD(Y) .

The squared multiple correlation coefficient of group p is,

ρ2p = 1 − var(D) (6)

By putting Equation 5 into Equation 6 we have

ρ2p =

k∑i=1

γ ∗i

2 + 2∑∑

i�=j

γ ∗i γ ∗

j �∗ij (7)

Figure 1b shows the half-transformed model of M1. Eachobserved variable is regressed on a dummy latent variable1

(DLV) such that (k + 1) DLVs, F1, F2, . . . , Fk, and FYare added. The function of these DLVs is to manipulate thevariance of the variables so as to reparameterize the model.Factor loadings, λi from Fi to Xi are free to be estimated.In addition, the path coefficient from Fi to FY is fixed atai depending on the sign of γ i

∗. If γ i∗ > 0, then ai = 1;

otherwise, ai = –1. The model equation for FY is as follows:

1 The function of DLV is similar to Rindskopf’s concept of a phantomvariable. However, unlike Rindskopf’s (1984) original definition of a phan-tom variable, which is a latent variable with no observed indicators, a DLVhas other variables loaded on it. Chan (2007) used the term DLV to denotethe variable that is used to factorize the original mediator in the sequentialmodel fitting method. We use the term DLV here because the latent vari-ables in the transformed model have the original observed variables loadedon them.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

SQUARED MULTIPLE CORRELATION COEFFICIENTS 229

FY =k∑

i=1

aiFi + D (8)

and the variance of FY is,

var(FY) = var

(k∑

i=1

aiFi + D

)

=k∑

i=1

φii + 2∑∑

i�=j

(ai)(aj)φij + var(D)

= g(θ ) + var(D)

(9)

where øii is the variance of Fi; øij is the covariance betweenFi and Fj; θ is a vector of unknown model parameters;

and g(θ ) =k∑

i=1φii + 2

∑∑i�=j

(ai)(aj)φij is defined as the total

variance and covariance due to the antecedent variables.If var(FY) is fixed at 1, Equation 9 will be equivalent toEquation 5 with φii = γ ∗2

i and φij = ∣∣γ ∗i

∣∣ ∣∣γ ∗j

∣∣�∗ij given that

ai = 1 or –1 and it always shares the same sign as γ i∗.

Therefore, ρ2p of the original model will be the same as g(θ )

in this transformed model. Hence, the rest of the task is tofurther reparameterize the model such that g(θ ) appears as asingle model parameter in the final transformed model and atthe same time fix the variance of FY at 1.0.

To do this, we further transform the model into the finalR2 model, M2 (as shown in Figure 1c) by regressing thedisturbance term, D on a phantom variable, F999 with unitvariance and a pair of phantom variables, FR and FR.’ Welabel the structure formed by FR and FR’ as the R2 structure.The R2 structure has the following four properties: (a) boththe variances of FR and FR’ are fixed at 0; (b) the path coeffi-cient from FR to D is fixed at 1; (c) the path coefficient fromFR’ to D is fixed at –0.5; and (d) the covariance between FRand FR’ is constrained to be equal to g(θ ). Notice that g(θ ) issimply a linear function of the model parameters and hencethe proposed method does not require the specification of anynonlinear constraints. In addition, the variance of F999 andthe path leading from F999 to D are fixed at 1.0.

Because var(F999) = 1; var(FR) = var(FR’) = 0; and thecovariance between FR and FR’ is constrained to be equal tog(θ ), the variance of D becomes

D = F999 + FR − 0.5FR′

var(D) = var(F999) + var(FR)

+ (−0.5)2 var(FR′) + 2(1)(−0.5)g(θ )

var(D) = 1 − g(θ )

(10)

By substituting Equation 10 into Equation 9, we have thevariance of FY fixed at 1.0 nonstochastically:

var(FY) = g(θ ) + (1 − g(θ )) = 1 (11)

Hence, g(θ ) is equivalent to the ρ2p of the original model, M1,

and more important, it appears as a single model parameter;that is, the covariance between FR and FR’ in M2.

To summarize, three important criteria need to beobserved when one performs model transformation. First,the variance of FY in M2 must be fixed at 1.0 nonstochasti-cally. Second, the ρ2

p , which is defined by g(θ ), must appearas a single model parameter in M2. Third, M2 must havethe same implied covariance structure as the original model.Once the original model is successfully transformed into theR2 model at Stage 1, it is straightforward for us to comparethe ρ2 in different groups by using the LR test at Stage 2.

A REAL EXAMPLE

A real example based on the data from the Organizationfor Economic Cooperation and Development (OECD)Programme for International Student Assessment (PISA)2006 (OECD, 2009)2 is given to illustrate the proposedmethod using EQS (Bentler, 1995). Readers who are inter-ested in working on this example can refer to Appendix Afor the EQS program codes. Cases from three countries—Hong Kong, Japan, and Korea—are selected from the PISA2006 data set on four variables. They are, namely, generalinterest in science (V1), enjoyment of science (V2), scienceself-efficacy (V3), and science ability (V4). The sample sizesare 4,625 for Hong Kong, 5,943 for Japan, and 5,151 forKorea. Table 1 summarizes the sample covariance matricesof the variables.

Stage 1: Model Specification of M1 and ModelTransformation Into M2

A regression model with three predictors is considered.Specifically, we compared if general interest in science(V1), enjoyment of science (V2), and science self-efficacy(V3) as a set of predictors about students’ views on sci-ence provided the same degree of predictive power onstudents’ science ability (V4) in Hong Kong (HKG), Japan(JPN), and Korea (KOR). Therefore, we tested the hypoth-esis, H0: ρ2

HKG = ρ2JAP = ρ2

KOR. Figure 2a shows the originalmodel, M1. A multisample analysis was performed usingEQS6.1 for Windows. Because M1 is a saturated modelwith 0 degrees of freedom, it has a perfect fit with model

2The PISA is a triennial international assessment of 15-year-old schoolchildren’s capabilities of reading literacy, mathematics literacy, and scienceliteracy. The database can be freely accessed through the PISA Web page(www.pisa.oecd.org). In this article, data from PISA collected in 2006 isused. PISA 2006 has been administered in 57 countries or economies(OECD, 2009). We have selected cases from three countries—namelyHong Kong, Japan, and Korea—from the PISA 2006 data, and four variableshave been used in this article.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

230 KWAN AND CHAN

TABLE 1Sample Covariance Matrices for Hong Kong, Japan and Korea from Programme for International Student

Assessment 2006 Data (Organization for Economic Cooperation and Development, 2009)

General Interest inScience (V1)

Enjoyment ofScience (V2)

ScienceSelf-Efficacy (V3)

Science Ability(V4)

Hong Kong (N = 4,625)V1 0.9451V2 0.6006 0.7977V3 0.4326 0.3779 0.8956V4 27.3990 28.2320 31.2722 8176.0021Japan (N = 5,943)V1 1.0453V2 0.6926 1.0777V3 0.5027 0.4524 0.9583V4 34.2501 35.2189 30.6472 9666.8658Korea (N = 5,151)V1 0.9271V2 0.6338 1.0007V3 0.4088 0.3902 0.8031V4 31.6266 37.3062 30.9021 8187.6921

(a)

(b)

FIGURE 2 Sequence of models in Programme for International Student Assessment (PISA) example. (a) Original model (M1). (b) R2 Model (M2). V1 =general interest in science, V2 = enjoyment of science, V3 = science self-efficacy, V4 = science ability. ø67 = ø11 + ø22 + ø33 + 2ø12 + 2ø13 + 2ø23.

chi-square, χ2 = 0. The estimated R2 coefficients forHong Kong (R2

HKG), Japan (R2JPN), and Korea (R2

KOR) are .178,.163, and .224, respectively.

We followed the general framework to transform M1 intothe R2 model, M2 (Figure 2b). Each observed variablewas regressed on a DLV (F1 to F4). As the standardizedregression coefficients from V1, V2, and V3 to V4 (i.e.,γ ∗

1 , γ ∗2 and γ ∗

3 ) were all positive in M1, the paths from F1,

F2, and F3 to F4 were all fixed at ai = 1.0 (i = 1, 2, and3). F5 was the disturbance term of F4.3 F999 was the phan-tom variable with variance of 1.0, and the path from F999 to

3In EQS, the disturbance term can appear only as an exogenous variable,which cannot be regressed on other variables. To solve this problem, werenamed the disturbance term as F5, an ordinary latent factor.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

SQUARED MULTIPLE CORRELATION COEFFICIENTS 231

TABLE 2Summary of the R2 Estimates and Their Estimated Standard Errors of the Original Model (M1) in Programme for International

Student Assessment (PISA) Example by Analysis Using Different Structural Equation Modeling Programs and Approaches

Program EQS 6.1 Mplus 5.2 LISREL8.8

Method Reparameterization Built-In Phantom Variable

R2/ φ67 Est SE Est SE Est SE

Hong Kong 0.178 0.010 0.178 0.010 0.178 0.010Japan 0.163 0.009 0.163 0.009 0.163 0.009Korea 0.224 0.010 0.224 0.010 0.224 0.010

Note. Est. = parameter estimate.

TABLE 3Models Summary in the Programme for International Student Assessment (PISA) Example

Models φ67(HKG) φ67

(JAP) φ67(KOR) χ2 Df p

M1 .178 (—) .163 (—) .224 (—) 0 0 —M2

Unconstrained .178 (.010) .163 (.009) .224 (.010) 0 0 —Constrained under

H0 : ρ2HKG = ρ2

JAP = ρ2KOR .187 (.006) .187 (.006) .187 (.006) 21.480 2 <.001

H0 : ρ2HKG = ρ2

JAP .170 (.007) .170 (.007) .224 (.010) 1.334 1 .248H0 : ρ2

JAP = ρ2KOR .178 (.010) .191 (.007) .191 (.007) 20.496 1 <.001

Note. Estimated standard errors of the parameter estimates are in parentheses.

F5 was fixed at 1.0. F6 = FR and F7 = FR’ formed the R2

structure. Hence, variances of F6 and F7 were fixed at zeroand the paths from F6 and F7 to F5 were fixed at 1.0 and–0.5, respectively.

We then performed a multisample analysis based onM2 using EQS.4 The covariance between F6 and F7, ø67,in each group was constrained to be equal to the sum ofthe variances of F1 to F3 and two times covariances amongF1, F2, and F3 (i.e., ø67 – (ø11 + ø22 + ø33 + 2ø12 +2ø13 +2ø23) = 0). Therefore, three linear constraints on ø67

(one for each of the three countries: Hong Kong, Japan, andKorea) were imposed on the model altogether. Again, themodel has a perfect fit with the model chi-square, χ2 = 0,df = 0. Table 2 shows the parameter estimates for ø67 andtheir SEs in M2 (under the heading “reparameterization”).By comparing the estimated ø67 and their SEs with the R2 andtheir SEs reported by Mplus (with a built-in function) andLISREL (by using the phantom variable approach5), theyare perfectly comparable with each other. In other words, the

4 The R2 model failed to converge initially with the default start valuesin EQS. The problem was solved by providing better start values to the algo-rithm. As a general guideline, one can first fit the original model to obtainγ ∗

i ,the standardized regression coefficients of Vi, �∗

ij, the correlation between

Vi and Vj, and R2 for each sample and use them to compute the start valuein the transformed model. Start value for var(Fi), cov(Fi, Fj), and cov(FR,

FR’) can be set to be equal to γ ∗2i ,∣∣γ ∗

i

∣∣ ∣∣∣γ ∗j

∣∣∣�∗ij, and R2, respectively.

5By following the phantom variable approach, we created new param-eters to define the R2 for each group (using AP keyword) in LISREL. The

original model has been successfully transformed into the R2

model, and the estimated ø67 in each group that appears inM2 is equivalent to the R2 of the original model, M1 in thecorresponding group.

Stage 2: Testing the Equality of R2 Using LR Test

To test the equality of the ρ2 in the three groups, we fitteda constrained model under H0: ρ2

HKG = ρ2JAP = ρ2

KOR byimposing two linear equality constraints: (a) φ

(HKG)67 = φ

(JAP)67

and (b) φ(JAP)67 = φ

(KOR)67 on M2. The model chi-square was

χ2(2) = 21.480, p < .001. Table 3 shows the model chi-squares and the parameter estimates of the constrained andunconstrained models. We compared the chi-square of theconstrained model with that of the unconstrained model. TheLR test gave �χ2 = 21.480 – 0 = 21.480, �df = 2 – 0 = 2,p < .001, suggesting that there was a significant differenceamong the ρ2 in Hong Kong, Japan, and Korea. Hence,the predictive powers of the set of predictors (i.e., generalinterest, enjoyment, and self-efficacy in science) on scienceability were different in these three countries. As a post-hoccomparison, we further evaluated the significance of eachequality constraint in a pairwise fashion. The hypothesesH0: ρ2

HKG = ρ2JAP and H0: ρ2

JAP = ρ2KOR were tested separately

by using the LR test. The LR test results show that there is no

estimated SEs of the new parameters, which are equivalent to the estimatedSEs of the corresponding R2, are reported.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

232 KWAN AND CHAN

significant difference in ρ2 between Hong Kong and Japan(d = 0.016, �χ2 = 1.334, �df = 1, p > .05). In contrast,there is a significant difference in ρ2 between Japan andKorea (d = 0.061, �χ2 = 20.496, �df = 1, p < .001).As compared with the Wald test results based on the analysisby LISREL, the two tests give the same conclusion aboutthe two pairs of R2 (see Table 4). This suggests that the LRtest behaves similarly to the Wald test used by the phantomvariable approach for comparing R2. LISREL, however, failsto provide the overall test for H0: ρ2

HKG = ρ2JAP = ρ2

KOR.

A SIMULATION STUDY

To further evaluate the accuracy of the proposed method,a simulation study was conducted to compare the perfor-mance of the proposed method with the traditional Olkin andFinn (1995) methods and Cheung’s (2009) phantom variableapproach on comparing the ρ2 in different groups.

Model Specification

A multiple regression model of Y (p)on X(p) = (X1(p), X2

(p),. . . , Xk

(p)) for group p = 1, 2, 3 was considered. In thisstudy, we used a correlation model so that both the crite-rion and the predictors were assumed to be random. Thecorrelation model is commonly regarded as a more reason-able model in multiple regression analysis because randompredictors are more commonly used than fixed predictors insocial sciences research (Stevens, 2002). Consequently, thevariables have a joint distribution with covariance matrix

(p) =

(σ

(p)YY σ

(p)XY

′

σ(p)XY σ

(p)XX

)(12)

where σ(p)YY is the variance of Y (p), σ(p)

XX is the k × k covariancematrix of X(p), and σ

(p)XY is the k × 1 vector of covariance

between X(p) and Y (p).To generate simulated data, the values of the population

parameters in Equation 12 were specified. First, without lossof generality, the means and variances of the variables werefixed at 0.0 and 1.0, respectively. Second, the covariancesamong the predictors were all fixed at 0.3 as is typical in

social sciences research. Finally, a common covariance, σ (p)xy ,

was assumed between each of the predictors and the criterionin group p so that σ

(p)XY = 1σ (p)

xy , where 1 is a k × 1 unit vector.Following Johnson and Wichern’s (2002) results, we have

ρ2p = σ

(p)XY

′ (p)XX

−1σ

(p)XY

σ(p)YY

= σ (p)xy 1′ (p)−1

XX 1 (13)

Hence, the value of σ(p)xy can be determined accordingly once

ρ2p and

(p)XX have been fixed.

As the purpose here is to check if the proposed methodbehaves in the same way as any other asymptotic methodsand it is expected that the results are similar in different sim-ulation conditions, for simplicity, we only considered onesimulation condition. First, a regression model with threepredictors (k = 3) was used. Second, sample sizes werefixed to be equal across the three groups so that N1 = N2

= N3 = 60 to result in a sample size variable (N/k) ratioof 20. Finally, the population multiple correlation coeffi-cients of Group 1 and Group 2 were set to be equal withρ2

1 = ρ22 = 0.5 whereas, that of Group 3 was set as ρ2

3 =0.25. These settings reflect a large to very large effect sizein multiple regression analysis suggested by Cohen (1988).This model condition was selected because this is typical insocial sciences research and the findings would be relevant tothe applied settings. Table 5 gives the population covariancematrix of each group.

Procedures

Simple random sampling data with known population covari-ance matrices were generated with EQS 6.1 (Bentler, 1995);2,000 replications were generated. Based on the proposedmethod, Models M1 and M2 (unconstrained and con-strained) were fitted to the generated samples using EQS andLR tests were conducted to test the following hypotheses:

1. H0: ρ21 = ρ2

2 ;2. H0: ρ2

2 = ρ23 ;

3. H0: ρ21 = ρ2

2 = ρ23 .

TABLE 4Likelihood Ratio Test and Wald Test Results in the Programme for

International Student Assessment (PISA) Example

Likelihood Ratio Test Wald Testa

Hypothesis Testing �χ2 �df p Est SE (Est/SE)2 p

H0 : ρ2HKG = ρ2

JAP = ρ2KOR 21.480∗ 2 <.001 — — — —

H0 : ρ2HKG = ρ2

JAP 1.334 1 .248 .016 .013 1.329 .249H0 : ρ2

JAP = ρ2KOR 20.496∗ 1 <.001 .061∗ .013 20.485 <.001

Note. Est = parameter estimate of δ.aWald test results are based on the analysis by using LISREL 8.8.∗Result is significant at 5% significance level.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

SQUARED MULTIPLE CORRELATION COEFFICIENTS 233

TABLE 5Population Covariance Matrices for the Simulation Study

X1 X2 X3 Y

Group 1 = Group 2 X1 1.0000(ρ2

1 = ρ22 = 0.5; N1 = N2 = 60; k = 3) X2 0.3000 1.0000

X3 0.3000 0.3000 1.0000Y 0.5164 0.5164 0.5164 1.0000

Group 3 X1 1.0000(ρ2

3 = 0.25; N3 = 60; k = 3) X2 0.3000 1.0000X3 0.3000 0.3000 1.0000Y 0.3651 0.3651 0.3651 1.0000

TABLE 6Rejection Rates at 5% Significance Level

SEM Approaches Traditional Approaches

Hypothesis Proposed MethodPhantom Variable

Approach Olkin and Finn’s ACIOlkin and Finn’s

Fz (R2)

1. H0 : ρ21 = ρ2

2 6.0% 6.2% 6.5% 6.4%2. H0 : ρ2

2 = ρ23 40.5% 43.0% 43.7% 41.8%

3. H0 : ρ21 = ρ2

2 = ρ23 41.2% — — —

Note. Rejection rates were based on the 1,742 valid replications. ACI = asymptotic confidence interval.

In addition, Hypotheses 1 and 2 were tested by usingthe phantom variable approach (Cheung, 2009) with anal-ysis using LISREL and Olkin and Finn’s (1995) ACI andFZ(R2) methods based on the generated data. The numbersof replications found significance at α = 5% were recorded.

To evaluate the performance of the proposed method, therejection rates were computed by dividing the numbers ofreplications found significant at α = 5% with the total num-ber of replications. It is expected that the proposed methodbehaves similarly to other methods so that the rejection ratesfor Hypotheses 1 and 2 are similar across different meth-ods. As the phantom variable approach and Olkin and Finn’smethods are not applicable for testing Hypothesis 3, we onlypresent the rejection rate of the proposed method.

Results

In this simulation study, replications that resulted in any neg-ative regression estimates were excluded from the analysis.6

In addition, Model M2 failed to converge in some of thereplications. As a result, only 1,742 successful replicationswere used in computing the rejection rates.

6EQS requires us to use the same model specification in all the replica-tions when a simulation study is conducted. As the population γ i

∗ (i = 1,2, and 3) were all positive, we preset the path coefficients from F1, F2, andF3 to F4 in M2 as ai = 1 and the generated data were fit to this M2 modelin all the replications. As a result, some of the replications failed becauseof misspecification of M2 when the generated data produced any negativestandardized regression coefficients in M1. For the sake of convenience,we simply excluded all the replications that gave any negative standardizedregression coefficients in M1 from subsequent analysis.

As shown in Table 6, under the condition that the nullhypothesis was true, the rejection rate for H0: ρ2

1 = ρ22 of

the proposed method was close to those of other methods.This suggests that the proposed method performs similarlyto other methods. In addition, the rejection rate of 6% forthe proposed method is close to the nominal Type I errorrate.

When H0: ρ22 = ρ2

3 was tested, all of the methods behavedsimilarly with rejection rates around 40%, which was theestimated power of the tests. The rejection rate for testingH0: ρ2

1 = ρ22 = ρ2

3 based on the proposed method was 41.2%.As other methods failed to test the preceding hypothesis, nocomparison of the performance of different methods could bemade. To conclude, the results of the simulation study sug-gest that our proposed method, in general, performs equallywell as other methods for testing the equality of ρ2 in atwo-group condition.

DISCUSSION

In this article, a method for comparing squared multiplecorrelation coefficients in different groups using SEM isproposed. A real example has been given to illustrate theimplementation of the proposed method. In the example,our method performed accurately in reparameterizing theR2 coefficients at Stage 1. It gave the same R2 and stan-dard errors, as compared with those reported by Mplus andLISREL (using the phantom variable approach). At Stage 2,the LR test could be employed as a routine step to comparethe ρ2 in different groups.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

234 KWAN AND CHAN

Theoretically speaking, the LR test and the Wald testemployed by the phantom variable approach address thesame question, and they are asymptotically equivalent underthe same null hypothesis (Chou & Bentler, 1990; Satorra,1989). We compared the LR test with the Wald test results inthe example and found that the p values (Table 4) reported bythe LR test and the Wald test were highly comparable. Thissuggests that the LR test at Stage 2 behaves similarly to theWald test when comparing ρ2.

To further verify the accuracy of the proposed method, asimulation study was conducted to compare the performanceof various methods when comparing ρ2 in different groups.The rejection rates based on the LR test were similar to thosebased on the Wald test and the Olkin and Finn’s methodswhen two different hypotheses were tested. This indicatesthat the proposed method performs similarly to other meth-ods. Hence, the results further support the adequacy of theproposed method for making statistical inferences about theequality of ρ2 across groups.

There are several distinguishing features of the proposedmethod. First, unlike other methods that are proposed forcomparing ρ2 in two groups, the proposed method is amore general one as it is capable of comparing three ormore between-groups ρ2 simultaneously. By using the LRtest at Stage 2, we can test the equality of ρ2 in m differ-ent groups by simply imposing (m – 1) linear cross-groupequality constraints on the R2 and comparing the chi-squarestatistics between the constrained and unconstrained R2 mod-els. As mentioned at the beginning, there are many situationsin which researchers have to consider multiple-groups com-parisons in their studies. The proposed method provides asimple and flexible way for researchers to carry out thiscomparison, whereas existing methods fail to do so.

Second, the method is compatible with most SEM soft-ware programs on the market. As mentioned earlier, thenonlinear constraint is a necessary feature of the phantomvariable approach. Therefore, users must use a SEM pro-gram that supports model fitting with nonlinear constraintsto implement this method. In contrast, our proposed methodapplies the model reparameterization technique to convertthe squared multiple correlation into a single model param-eter. Consequently, our method only requires basic standardfunctions, which are supported by most SEM software pro-grams. Apart from using EQS for the implementation of theproposed method as demonstrated here, the method workswell with other programs, such as LISREL. Researchers arefree to choose their favorite SEM programs for implementingthe proposed method.

In relation to this, another advantage of the proposedmethod is that one can keep the use of programming toa minimum. One disadvantage of the phantom variableapproach is researchers have to fully understand the func-tional relationships among the model parameters before theycan specify the nonlinear constraints on the new parame-ters correctly. In contrast, our proposed method can avoid

this problem because researchers can use graphical pro-gramming capabilities possessed by some SEM softwareprograms (e.g., EQS) to implement the proposed method andavoid the complicated syntax. In fact, many SEM beginnersprefer SEM software programs such as EQS because of theirwell-designed graphical user interface (e.g., Kline, 1998).Even SEM beginners can manage to implement the pro-posed method because they can use the graphical interfaceand simply follow the general framework of model transfor-mation to specify the models of interest without going intothe mathematical details.

Finally, the proposed method can also be used for com-paring the adjusted squared multiple correlation coefficient,R2

c ,

R2c = 1 − N − 1

N − k − 1(1 − R2)

= N − 1

N − k − 1R2 − k

N − k − 1

(14)

where N is the sample size and k is the number of pre-dictors. As R2 is known to be a biased estimator of ρ2,some researchers suggest using R2

c when the comparison ismade. For example, Algina and Keselman (1999) observeda decline in performance of Olkin and Finn’s method forunequal ρ2s in their simulation study. They postulated thatif such a decline is due to the bias in R2, the performancecan be improved by using R2

c instead. The proposed methodcan be adapted to make multiple-group comparisons of R2

cby simply modifying the model specification of M2 model atStage 1 (see Appendix B).

CONCLUDING REMARKS

There are several remarks worth noting. First, it shouldbe noted that the overall significance of the testH0: ρ2

1 = ρ22 = . . . = ρ2

m does not give any informationabout the equality of specific regression coefficients acrossgroups. Even if H0: ρ2

1 = ρ22 = . . . = ρ2

m is true, it is stillpossible for one or more of the regression coefficients tovary across the groups or vice versa. For example, wefound that there was no significant difference in ρ2 betweenHong Kong and Japan in the PISA example. Nevertheless,when we look at the regression coefficients of generalinterest in science (V1), the standardized estimate forHong Kong is 2.4 times smaller than that of Japan (0.057 vs.0.139) even though ρ2

HKG=ρ2JAP. Therefore, the proposed

method should not be regarded as an omnibus procedure fordeciding whether to compare the regression coefficients ofspecific variables across groups (Algina & Keselman, 1999).

Second, a multiple regression model with three predic-tors has been used in this article to demonstrate the proposedmethod. Model transformation could become tedious whenmore predictors are involved. However, the general principle

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

SQUARED MULTIPLE CORRELATION COEFFICIENTS 235

of model transformation remains unchanged, regardless ofthe fact that more DLVs are needed to reparameterize theoriginal model. In addition, the proposed method can beextended to models with latent variables. Nevertheless, theproposed model might not work well in other model condi-tions, such as models with mediators. Further investigation isneeded to evaluate the effectiveness of the proposed methodin these conditions.

Finally, given that the LR test employed in the proposedmethod is based on asymptotic theories, the accuracy ofthe proposed method could vary depending on the samplesize and distribution of data. The simulation results suggestthat the proposed method behaves similarly to other exist-ing asymptotic methods. In relation to this, future researchcan also be done to explore the possibility of integrating theproposed method and bootstrapping (Bollen & Stine, 1993;Yung & Bentler, 1996), which does not rely on any distri-butional assumptions about the data on the multiple-groupcomparison of squared multiple correlation coefficients.

REFERENCES

Alf, E. F., & Graf, R. G. (1999). Asymptotic confidence limit for the differ-ence between two squared multiple correlations: A simplified approach.Psychological Methods, 4, 70–75.

Algina, J., & Keselman, H. J. (1999). Comparing squared multiple corre-lation coefficients: Examination of a confidence interval and a test ofsignificance. Psychological Methods, 4, 76–83.

Arbuckle, J. L. (2007). AMOS 16.0 user’s guide. Chicago, IL: SPSS.Bentler, P. M. (1995). EQS structural equations program manual. Encino,

CA: Multivariate Software.Bollen, K. A., & Stine, R. (1993). Bootstrapping goodness-of-fit measures

in structural equation models. In K. A. Bollen & J. S. Long (Eds.),Testing structural equation models (pp. 111–135). Newbury Park, CA:Sage.

Chan, W. (2007). Comparing indirect effects in structural equationmodeling: A sequential model fitting method using covariance-equivalentspecifications. Structural Equation Modeling, 14, 326–346.

Chan, W. (2009). Bootstrap standard errors and confidence inter-vals for the difference between two squared multiple correla-tion coefficients. Educational and Psychological Measurement, 69,566–584.

Cheung, M. W. L. (2009). Constructing approximate confidence inter-vals for parameters with structural equation models. Structural EquationModeling, 16, 267–294.

Cheung, M. W. L., & Chan, W. (2005). Meta-analytic structural equationmodeling: A two-stage approach. Psychological Methods, 10, 40–64.

Chou, C. P., & Bentler, P. M. (1990). Model modification in covariancestructure modeling: A comparison among likelihood ratio, Lagrangemultiplier, and Wald tests. Multivariate Behavioral Research, 25,115–136.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nded.). Hillsdale, NJ: Erlbaum.

Cole, D. A., Maxwell, S. E., Arvey, R., & Salas, E. (1993). Multivariategroup comparison of variable systems: MANOVA and structural equationmodeling. Psychological Bulletin, 114, 174–184.

Efron, B., & Tibshirani, R. (1993). An introduction to the bootstrap. NewYork, NY: Chapman & Hall.

Fan, X. (1997). Canonical correlation analysis and structural equationmodeling: What do they have in common? Structural Equation Modeling,4, 65–79.

Gajjar, A. V. (1967). Limiting distributions of certain transformations ofmultiple correlation coefficients. Metron, 26, 189–193.

Hershberger, S. L. (2003). The growth of structural equation modeling:1994–2001. Structural Equation Modeling, 3, 93–104.

Johnson, R. A., & Wichern, D. W. (2002). Applied multivariate statisticalanalysis (5th ed.). Upper Saddle River, NJ: Prentice Hall.

Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8: A user’s reference guide.Chicago, IL: Scientific Software International.

Kline, R. B. (1998). Software review: Software programs for struc-tural equation modeling: Amos, EQS, and LISREL. Journal ofPsychoeducational Assessment, 16, 343–364.

Kwan, J. L. Y., & Chan, W. (2011). Comparing standardized coefficientsin structural equation modeling: A model reparameterization approach.Behavior Research Methods, 43, 730–745.

Muthén, L. K., & Muthén, B. O. (2007). Mplus user’s guide (5th ed.). LosAngeles, CA: Muthén & Muthén.

Olkin, I., & Finn, J. D. (1995). Correlation redux. Psychological Bulletin,118, 155–164.

Organization for Economic Cooperation and Development. (2009). PISAdata analysis manual: SPSS (2nd ed.). Paris, France: OECD.

Rindskopf, D. (1984). Using phantom and imaginary latent variables toparameterize constraints in linear structural models. Psychometrika, 49,37–47.

Satorra, A. (1989). Alternative test criteria in covariance structure analysis:A unified approach. Psychometrika, 54, 131–151.

Stevens, J. (2002). Applied multivariate statistics for the social sciences (4thed.). Mahwah, NJ: Lawrence Erlbaum.

Tremblay, P. F., & Gardner, R. C. (1996). On the growth of structural equa-tion modeling in psychological journals. Structural Equation Modeling,3, 93–104.

Yung, Y., & Bentler, P. M. (1996). Bootstrapping techniques in analy-sis of mean and covariance structures. In G. A. Marcoulides & R. E.Schumacker (Eds.), Advanced structural equation modeling: Issues andtechniques (pp. 195–226). Mahwah, NJ: Erlbaum.

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

236 KWAN AND CHAN

APPENDIX A

EQS PROGRAM CODES FOR THE REAL EXAMPLE

Program Codes for Original Model M1

/TITLEOriginal Model - HKG/SPECIFICATIONSVARIABLES=4; CASES=4625; GROUPS=3; METHOD=ML; ANALYSIS=COVARIANCE; MATRIX=COVARIANCE;/LABELSV1=V1; V2=V2; V3=V3; V4=V4;/EQUATIONSV4 = ∗V1 + ∗V2 + ∗V3 + E4;/VARIANCESV1 to V3 = ∗; E4 = ∗;/COVARIANCESV1,V2 = ∗; V1,V3 = ∗; V2,V3 = ∗;/MATRIX0.94510.6006 0.79770.4326 0.3779 0.895627.3990 28.2320 31.2722 8176.0021/END/TITLEOriginal Model - JPN/SPECIFICATIONSVARIABLES=4; CASES=5943; METHOD=ML; ANALYSIS=COVARIANCE; MATRIX=COVARIANCE;/LABELSV1=V1; V2=V2; V3=V3; V4=V4;/EQUATIONSV4 = ∗V1 + ∗V2 + ∗V3 + E4;/VARIANCESV1 to V3 = ∗; E4 = ∗;/COVARIANCESV1,V2 = ∗; V1,V3 = ∗; V2,V3 = ∗;/MATRIX1.04530.6926 1.07770.5027 0.4524 0.958334.2501 35.2189 30.6472 9666.8658/END/TITLEOriginal Model - KOR/SPECIFICATIONSVARIABLES=4; CASES=5151; METHOD=ML; ANALYSIS=COVARIANCE; MATRIX=COVARIANCE;/LABELSV1=V1; V2=V2; V3=V3; V4=V4;/EQUATIONSV4 = ∗V1 + ∗V2 + ∗V3 + E4;/VARIANCESV1 to V3 = ∗; E4 = ∗;/COVARIANCESV1,V2 = ∗; V1,V3 = ∗; V2,V3 = ∗;/MATRIX0.92710.6338 1.00070.4088 0.3902 0.803131.6266 37.3062 30.9021 8187.6921/PRINTCOVARIANCE=YES; FIT=ALL; TABLE=EQUATION;/END

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

SQUARED MULTIPLE CORRELATION COEFFICIENTS 237

Program Codes for R2 Model M2

/TITLERˆ2 Model M2 - HKG/SPECIFICATIONSVARIABLES=4; CASES=4625; GROUPS=3; METHOD=ML; ANALYSIS=COVARIANCE; MATRIX=COVARIANCE;/LABELSV1=V1; V2=V2; V3=V3; V4=V4;/EQUATIONSV1 = ∗F1; V2 = ∗F2; V3 = ∗F3; V4 = ∗F4;F4 = F1 + F2 + F3 + F5; F5 = F999 + F6 - 0.5F7;/VARIANCESF1 = .025∗; F2 = .05∗; F3 = .05∗; F6 to F7 = 0; F999 = 1;/COVARIANCESF2,F1 = .0125∗; F3,F1 = .0125∗; F3,F2 = .0125∗; F7,F6 = 0.2∗;/MATRIX0.94510.6006 0.79770.4326 0.3779 0.895627.3990 28.2320 31.2722 8176.0021/END/TITLERˆ2 Model M2 - JPN/SPECIFICATIONSVARIABLES=4; CASES=5943; METHOD=ML; ANALYSIS=COVARIANCE; MATRIX=COVARIANCE;/LABELSV1=V1; V2=V2; V3=V3; V4=V4;/EQUATIONSV1 = ∗F1; V2 = ∗F2; V3 = ∗F3; V4 = ∗F4;F4 = F1 + F2 + F3 + F5; F5 = F999 + F6 - 0.5F7;/VARIANCESF1 = .025∗; F2 = .05∗; F3 = .05∗; F6 to F7 = 0; F999 = 1;/COVARIANCESF2,F1 = .0125∗; F3,F1 = .0125∗; F3,F2 = .0125∗; F7,F6 = 0.2∗;/MATRIX1.04530.6926 1.07770.5027 0.4524 0.958334.2501 35.2189 30.6472 9666.8658/END/TITLERˆ2 Model M2 - KOR/SPECIFICATIONSVARIABLES=4; CASES=5151; METHOD=ML; ANALYSIS=COVARIANCE; MATRIX=COVARIANCE;/LABELSV1=V1; V2=V2; V3=V3; V4=V4;/EQUATIONSV1 = ∗F1; V2 = ∗F2; V3 = ∗F3; V4 = ∗F4;F4 = F1 + F2 + F3 + F5; F5 = F999 + F6 - 0.5F7;/VARIANCESF1 = .025∗; F2 = .05∗; F3 = .05∗; F6 to F7 = 0; F999 = 1;/COVARIANCESF2,F1 = .0125∗; F3,F1 = .0125∗; F3,F2 = .0125∗; F7,F6 = 0.2∗;/CONSTRAINTS!constrain phi67(HKG)(1,F7,F6)-(1,F1,F1)-(1,F2,F2)-(1,F3,F3)-2(1,F2,F1)-2(1,F3,F1)-2(1,F3,F2)=0;!constrain phi67(JPN)(2,F7,F6)-(2,F1,F1)-(2,F2,F2)-(2,F3,F3)-2(2,F2,F1)-2(2,F3,F1)-2(2,F3,F2)=0;!constrain phi67(KOR)(3,F7,F6)-(3,F1,F1)-(3,F2,F2)-(3,F3,F3)-2(3,F2,F1)-2(3,F3,F1)-2(3,F3,F2)=0;!constrain ‘phi67(HKG) = phi67 (JPN)’(1,F7,F6)-(2,F7,F6)=0;!constrain ‘phi67(JPN) = phi67 (KOR)’(2,F7,F6)-(3,F7,F6)=0;/MATRIX

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014

238 KWAN AND CHAN

0.92710.6338 1.00070.4088 0.3902 0.803131.6266 37.3062 30.9021 8187.6921/PRINTCOVARIANCE=YES;FIT=ALL;TABLE=EQUATION;/END

APPENDIX B

M2 MODEL FOR THE COMPARISON OF ADJUSTED SQUARED MULTIPLE CORRELATION COEFFICIENTS (R2c)

The proposed method can be readily used for comparing the adjusted squared multiple correlation coefficients in differentgroups by simply modifying the R2 structure of the M2 model at stage 1. Figure B1 shows the modified M2 model. The modelspecification of this model is the same as that of the R2 model M2 shown in Figure 1c except (1) the covariance between FR

and FR’ is now constrained to be equal to g′(θ ) = N−1N−k−1 g(θ ) − k

N−k−1 where g(θ ) =k∑

i=1φii + 2

∑∑i�=j

(ai)(aj)φij; (2) the path

coefficient from FR to D is fixed at 1 − kN−1 ; (3) the variance of F999 is fixed at 1 − k

N−1 . Notice that both the path coefficientfrom FR to D and the variance of F999 are function of − k

N−1 . The variance of D becomes,

var(D) = var(F999) +(

1 − k

N − 1

)2

var(FR) + (−0.5)2 var(FR′) + 2(−0.5)

(1 − k

N − 1

)g′(θ )

= 1 − k

N − 1−(

N − k − 1

N − 1

)(N − 1

N − k − 1g(θ ) − k

N − k − 1

)

= 1 − g(θ )

(B1)

Hence, Equation B1 is equivalent to Equation 10 and the modified model is a covariance-equivalent model which shares thesame implied covariance matrix as the original model M1 and the R2 Model M2. Consequently, g(θ ) is the R2 of the originalmodel and the covariance between FR and FR’, g’(θ ), is reparameterized as R2

c . Because R2c appears as a single model parameter,

we can easily compare R2c in different groups by using the LR test.

FIGURE B.1 Modified M2 model for R2c . Parameters in parentheses are fixed parameters. g′(θ ) = N−1

N−k−1

(k∑

i=1φii + 2

∑∑i �=j

(ai )(aj )φij

)− k

N−k−1

Dow

nloa

ded

by [

Yor

k U

nive

rsity

Lib

rari

es]

at 0

9:51

13

Aug

ust 2

014