Factor Analysis of Data Matrices

PAUL HORST University of Washington

HOLT, RINEHART AND WINSTON, INC. New York Chicago San Francisco Toronto London

Contents

Preface

PART I. Introductory Background

1. The Role of Factor Analysis in Science 3

Data and Science 4 The Data Matrix 10 Combination of Matrix Categories 14 Objectives of Factor Analysis 16 Applications of Factor Analysis 24

2. Simple Matrix Concepts 27

Introduction 28 Matrix Notation 31 Kinds of Matrices 34 Transpose of a Matrix 40 Supermatrices 42 Transpose of the Supermatrix 46 Addition and Subtraction of Matrices 49 Vector Multiplication 52 Matrix Multiplication 57 Special Matrix Products 62

3. Matrix Structure and Solutions 74

Orthogonal Matrices 74 Rank of a Matrix 79 Finding the Rank of a Matrix 80 The Basic Structure of a Matrix 81 The Inverse of a Matrix 84 Inverse of a Supermatrix 88 The General Rank Reduction Theorem 90 Solving Linear Equations 92

X l l l

XIV CONTENTS

4. Matrix Factoring and Approximation 94

Essential Characteristics of Factor Analysis 94 The Rank Reduction Theorem 98 Basic Structure Solution 102 Indeterminateness of Factors 107 Kinds of Matrices 108 The Problem of Metrie 111

PART I I . Matrix Factoring Methods—A

5. The Centroid Method 114

Characteristics of the Method 114 Kinds of Solutions 116 Unity in the Diagonal 118 Optimizing the Diagonal Elements or Communality Estimates 125 Iterative Solution for Diagonals 129 Mathematical Proofs 131

6. Grouping Methods 137

Characteristics of the Grouping Method 137 Kinds of Grouping Methods 138 Group Centroid with Binary Vector 138 Group Centroid with Binary and Unit Vectors 144 The Multiple Group Method 147 Mathematical Proofs 152

7. Basic Structure Successive Factor Methods 156

Characteristics of the Methods 157 Kinds of Solutions 159 Solution with Residual Matrix 160 Solution without Residuais 167 Mathematical Proofs 173

8. Simultaneous Basic Structure Methods 178

Characteristics of the Solutions 178 Kinds of Solutions 180 The Rank Reduction Method 180 The Orthogonalization Method 185 Mathematical Proofs 191

CONTENTS XV

PART I N . Matrix Factoring Methods—B

9. Jacobi-Type Solutions 198

Characteristics of the Methods 198 Kinds of Methods 199 Simultaneous Method 200 Successive Method 205 Mathematical Proofs 206

10. Order Reduction Methods 211

Characteristics of the Methods 211 Kinds of Methods 212 The Partial Reduction Method 213 The Tridiagonal Solution 222 Mathematical Proofs 229

11. Solutions from Incomplete Covariance Matrices 237

Characteristics of the Solution 237 Kinds of Solutions 238 The Rxy Matrix 240 Simultaneous Solution for Two Submatrices 245 The Two-Matric Element, Two-Stage Solution 250 Mathematical Proofs 253

12. Factoring the Data Matrix 258

Characteristics of the Methods 258 Kinds of Methods 259 The Centroid Method 260 The Basic Structure Single Factor Method 266 Basic Structure with Simultaneous Factor Solution 273 Mathematical Proofs 278

PART IV. Categories, Origin, and Scale

13. The Problem of Origin 286

Kinds of Origin Problems 286 Ipsative Methods 291 Basic Structure and the Problem of Origin 295

XVI CONTENTS

Basic Structure of Raw Covariance Matrix from Correlation Matrix 296

The Normative Covariance Basic Structure from the Raw Covariance Basic Structure 303

The Ipsative Covariance Basic Structure from the Normative Covariance Basic Structure 305

The Normative Covariance Basic Structure from the Ipsative Basic Structure 307

Mathematical Proofs 310

14. Categorical Variations in Factor Analysis 315

Multicategory Sets 315 Consideration of Origin 323 Computational Considerations 324 Obverse Factor Solution with Standard Metrie 326 Mathematical Proof 330

15. The Problem of Scaling 333

Kinds of Scaling 333 Scaling by Attributes 335 The Communality Problem and Scaling 335 Characteristics of the Methods 336 Kinds of Solutions 337 Specificity Successive Factor Solution 338 The Specificity Factor Matrix Solution 342 The Specificity Progressive Factor Matrix Method 346 The Communality Successive Factor Method 348 The Communality Factor Matrix Solution 351 The Communality Progressive Factor Matrix Method 352 Mathematical Proofs 354

16. Image Analysis 361

Characteristics of the Methods 362 Kinds of Methods 364 The Image Covariance Matrix 365 The Image Correlation Matrix 369 The Independent Scale Matrix 370 The Optimal Residual or Anti-Image Matrix 372 Mathematical Proofs 375

CONTENTS XV11

PART V. Transformation Problems and Methods

17. Primary Factor Matrices from Hypotheses 384

Characteristics of the Hypothesis Methods 386 Kinds of Methods 388 The Multiple Group Factor Matrix 389 The Principal Axis Factor Matrix 394 The Arbitrary Factor Matrix 397 The Zero Partial Sum Transformation 401 The Orthogonal Transformation Matrix 406 Mathematical Proofs 410

18. Analytical Rotations 418

Characteristics of the Methods 420 Kinds of Methods 422 Successive Factor Varimax Solution 423 Simultaneous Factor Varimax Solution 428 Successive Factor General Varimax 430 Simultaneous Factor General Varimax 433 Mathematical Proofs 435

19. Direct Varimax Solutions 442

Characteristics of the Method 442 Kinds of Methods 444 The Successive Varimax Factor from the Correlation Matrix 446 The Simultaneous Varimax Matrix from the Correlation

Matrix 450 The Successive Factor Vector from the Data Matrix 453 The Simultaneous Factor Matrix from the Data Matrix 456 The Successive Factor General Varimax 459 The Simultaneous General Varimax 460 Mathematical Proofs 461

20. Factor Score Matrices 468

Introduction 468 Kinds of Factor Score Solutions 470 The Centroid Factor Score Matrix 471 The Multiple Group Factor Score Matrix 474 The Principal Axis Factor Score Matrix 476

XV111 CONTENTS

The Least Square Factor Score Matrix 478 The Image Analysis Factor Score Matrix 481 Mathematical Proofs 483

PART V I . Special Problems

21. General Factor Solutions 492

Introduction 492 General and Specific Simple Structure Factor Scores 496 The Factor-Score Covariance Matrix, and the Second-Order

Common and Specific Factor Loadings 497 The First-Order General and Common Factor Loading

Matrix 503 The First-Order General and Common Factor Score Matrix 505 Mathematical Proofs 507

22. Factor Analysis and the Binary Data Matrix 513

Introduction 513 Kinds of Solutions for Eliminating or Partialing Out the

Simplex 517 The Least Square Simplex Data Matrix 518 The Least Square Simplex Covariance Matrix 522 Computational Short Cut for the Simplex Covariance Matrix 525 Communality and the Simplex as a Special Case 526 Mathematical Proofs 527

23. Factor Analysis and Prediction 539

Kinds of Solutions 540 The Rand Reduction Solution 542 The Basic Structure Solution 545 The Nonbasic Solution 547 Increasing the Degrees of Freedom by Factor Analysis 551 Predictor Selection and Factor Analysis 554 Mathematical Proofs 558

24. Multiple Set Factor Analysis 565

Experimental Sources of the Models 565 Characteristics of the Methods 566 The Case of Two Sets 568

The Preliminary Orthogonalizations 569 The Maximum Correlation Method 571 The Rank One Approximation Method 576 The Oblique Maximum Variance Method 579 The Orthogonal Maximum Variance Method 582 Mathematical Proofs 584

Bibliography 596 Appendix—Fortran II Computer Programs 599 Index 719