Abstract

For multivariate observation X, factor analysis (FA) can be modeled as X = CF + UF, where CF and UF are abbreviations for common factor and unique factor parts, respectively. On the other hand, principal component analysis (PCA) can be modeled as X = CF + E, where E is an error part (and CF may be rewritten as a PC part). Here, E in the PCA model and UF in the FA model are distinguished in that inter-variable correlations are allowed in E, but not in UF. In this study, the model X = CF + UF + E is considered, which is a hybrid of the FA and PCA models. This study aims to present how the hybrid model X = CF + UF + E leads to hierarchical relationships among FA and PCA procedures. Here, the model is divided into random (R) and nonrandom (N) versions, with CF and UF treated as random variables in the R-version, but as nonrandom parameters in the N-version.
First, I focus on the N-version, whose least squares procedure is matrix decomposition FA (MDFA) (Adachi & Trendafilov, 2018). I discuss [1] how constraining MDFA leads to Stegeman’s (2016) completely decomposed FA (CDFA) and [2] how constraining CDFA leads to PCA. Thus, we have the hierarchy of MDFA > CDFA > PCA. Here, the procedure after > is a constrained variant of the one before >.
Then, I focus on the R-version of the hybrid model and prove the following three facts: [1] ten Berge and Kiers’ (1991) minimum rank FA (MRFA) is a procedure for estimating the parameters in the R-version; [2] By constraining the UF part to be null in MRFA, it leads to a random version of PCA (RPCA); [3] By constraining the E part to be uncorrelated among variables in RPCA, it leads to the prevalent FA (PrevFA) underlain by X = CF + UF. Thus, we have the hierarchy of MRFA > RPCA > PrevFA.
Finally, I argue that the above two hierarchies can be unified as MDFA > CDFA/MRFA > PCA/RPCA > PrevFA, from the equivalence of the CDFA solution to the MRFA one and that of the PCA solution to the RPCA one.

References
[1] Adachi, K., & Trendafilov, N. T. (2018). Some mathematical properties of the matrix decomposition solution in factor analysis. Psychometrika, 83, 407–424.
[2] Stegeman, A. (2016). A new method for simultaneous estimation of the factor model parameters, factor scores, and unique parts. Comput. Statist. Data Anal., 99,189–203.
[3] ten Berge, J. M. F. & Kiers, H. A. L. (1991). A numerical approach to the exact and the approximate minimum rank of a covariance matrix. Psychometrika, 56, 309–315.