View
38
Download
3
Category
Tags:
Preview:
DESCRIPTION
Canonical Correlation Analysis (CCA). CCA. This is it! The mother of all linear statistical analysis. When ? We want to find a structural relation between a set of independent variables and a set of dependent variables. CCA. When ? (part 2) - PowerPoint PPT Presentation
Citation preview
Canonical Correlation Analysis Canonical Correlation Analysis (CCA)(CCA)
CCACCA This is it!
The mother of all linear statistical analysis
When ? We want to find a structural relation between a set of independent variables and a set of
dependent variables.
and Canonical Coefficients
and Canonical Variates
ij ij
k k
f g
F G
1 11 1 12 2 1
1 11 1 12 2 1
2 21 1 22 2 2
21 1 22 2 2
1 1 2 2
1 1 2 2
First Canonical Variate Pairs
Second Canonical Variate Pairs2
p p
q q
p p
q q
m m m mp p
m m m mq q
F f x f x f x
G g x g x g x
F f x f x f x
G g x g x g x
F f x f x f x
G g x g x g x
th Canonical Variate Pairsm
T T0 and 0, i j i jF F G G i j
CCACCA When ? (part 2)
1. To what extend can one set of two or more variables be predicted or “explained” by another set of two or more variables?
2. What contribution does a single variable make to the explanatory power to the set of variables to which the variable belongs?
3. What contribution does a single variable contribute to predicting or “explaining” the composite of the variables in the variable set to which the variable does not belong?
4. What different dynamics are involved in the ability of one variable set to “explain” in different ways different portions of other variable set?
5. What relative power do different canonical functions have to predict or explain relationships?
6. How stable are canonical results across samples or sample subgroups?7. How closely do obtained canonical results conform to expected canonical
results?
CCACCA Assumptions
Linearity: if not, nonlinear canonical correlation analysis. Absence of multicollinearity: If not, Partial Least Squares (PLS) regression to reduce
the space. Homoscedasticity: If not, data transformation. Normality: If not, re-sampling. A lot of data: Max(p, q)20nb of pairs. Absence of outliers.
CCACCA Toy example
IVsIVs DVsDVs
= X
CCACCA Z score transformation
IV1IV1 DV2DV2IV1IV1 DV2DV2
= Z
CCACCA Canonical Correlation Matrix
T ( 1)pp pc
cp cc
n
R R
Z Z
R R
1 1cc cp pp pc R R R R R
CCACCA Relations with other subspace methods
1
Eigenvalues matrix
Eigenvectors matrix
B AV λV
λ
V
CCACCA Eigenvalues and eigenvectors decomposition
R =
T T T1 1 1 2 2 2 Min( ) Min( ) Min( )p q p q p q R V V V V V V
PCA
CCACCA Eigenvalues and eigenvectors decomposition
The roots of the eigenvalues are the canonical correlation values
2i ir
CCACCA Significance test for the canonical correlation
2
1
11 ln , with *
2
Where, (1 ) and =nb of variables Min( , )
m
m
m ii
p qN df p q
m p q
A significant output indicates that there is a variance share between IV and DV sets Procedure:
We test for all the variables (m=1,…,min(p,q)) If significant, we removed the first variable (canonical correlate) and test for the remaining ones (m=2,…, min(p,q) Repeat
CCACCA Significance test for the canonical correlation
2
1
11 ln , with *
2
Where, (1 ) and =nb of variables Min( , )
m
m
m ii
p qN df p q
m p q
Since all canonical variables are significant, we will keep them all.
CCACCA Canonical Coefficients
Analogous to regression coefficients
T1/ 2y yy y
B R V
Eigenvectors Correlation matrix of the
dependant variables
1
1
1
1,
where
x xx xy y
c
c cm
c
c cm
r r
r r
B R R Br
r
BY=
Bx=
CCACCA Canonical Variates
Analogous to regression coefficients
y y X Z B
x x Y Z B
CCACCA Loading matrices
Matrices of correlations between the variables and the canonical coefficients
x xx xA R B y yy yA R B
Ax
Ay
CCACCA Loadings and canonical correlations for both canonical variate pairs
Only coefficient higher than |0.3| are interpreted.
Loading Canonical correlation
CCACCA Proportion of variance extracted
How much variance does each of the canonical variates extract form the variables on its own side of the equation?
2
1
pixc
xci
apv
p
2
1
qiyc
yci
apv
p
First
Second
First
Second
CCACCA Redundancy
How much variance the canonical variates form the IVs extract from the DVs, and vice versa.
2crd pv r
Eigenvaluesrdyx
CCACCA Redundancy
How much variance the canonical variates form the IVs extract from the DVs, and vice versa.
Summary
The first canonical variate from IVs extract 40% of the variance in the y variable. The second canonical variate form IVs extract 30% of the variance in the y variable. Together they extract 70% of the variance in the DVs.
The first canonical variate from DVs extract 49% of the variance in the x variable. The second canonical variate form DVs extract 24% of the variance in the x variable. Together they extract 73% of the variance in the IVs.
CCACCA Rotation
A rotation does not influence the variance proportion or the redundancy.
= Loading matrix =
Recommended