## PCA / FA example 4: Davis. R-mode FA, eigenvalues

from davis’ “statistics and data analysis in geology”, we take the following extremely simple example (p. 502). his data matrix is
$D = \left(\begin{array}{ccc} 4&27&18\\ 12&25&12\\ 10&23&16\\ 14&21&14\end{array}\right)$
where each column is a variable. he now centers the data, by subtracting each column mean from the values in the column.
let me do that. i compute the column means…
${10,\ 24,\ 15}$
and subtract each one from the appropriate column, getting
$X = \left(\begin{array}{ccc} -6&3&3\\ 2&1&-3\\ 0.&-1&1\\ 4&-3&-1\end{array}\right)$
now let’s just follow him. he introduces the SVD (the singular value decomposition), by way of explanation, and we will use it, but not quite yet. he performs what’s called an R-mode factor analysis as follows.
compute $X^T X$:
$X^T X = \left(\begin{array}{ccc} 56&-28&-28\\ -28&20&8\\ -28&8&20\end{array}\right)$
then he computes the eigendecomposition. i get that the eigenvector matrix is…
$P = \left(\begin{array}{ccc} -2&0.&1\\ 1&-1&1\\ 1&1&1\end{array}\right)$
and that the diagonal matrix of eigenvalues is …
$\Sigma = \left(\begin{array}{ccc} 84&0.&0.\\ 0.&12&0.\\ 0.&0.&0.\end{array}\right)$
we run into our first notational quagmire. he has shown us the SVD $(X = u\ w\ v^{T})$ , and what we have here is $w^T w$, the eigenvalues of $X^T X$ (see “an SVD of X ~ eigenstructures of XTX and XXT”).
what he wants are the singular values w. no problem, you think: just take the square roots. sheesh, that’s what harman did, and maybe we’re beginning to see why.
unfortunately, davis is also using a 2×2 diagonal matrix instead of the 3×3 i get. to be more specific, i get w as a 4×3 and $w^T w$ as a 3×3. to be completely specific, i get w is…
$w = \left(\begin{array}{ccc} 2 \sqrt{21}&0.&0.\\ 0.&2 \sqrt{3}&0.\\ 0.&0.&0.\\ 0.&0.&0.\end{array}\right)$
and therefore $w^T w$ is…
$w^T w = \left(\begin{array}{ccc} 84&0.&0.\\ 0.&12&0.\\ 0.&0.&0.\end{array}\right)$
which is, as it should be, the 3×3 matrix of eigenvalues.
well, having said that he wants the square roots, i’ll give them to him, but i will modify his notation. i take $\Lambda$ to be a 3×3 diagonal matrix of square roots of eigenvalues, one of which is zero…
$\Lambda = \left(\begin{array}{ccc} 9.16515&0.&0.\\ 0.&3.4641&0.\\ 0.&0.&0.\end{array}\right)$
and i will use $\Lambda 2$ for his 2×2 matrix of nonzero square roots (which he called $\Lambda$).
$\Lambda 2 = \left(\begin{array}{cc} 9.16515&0.\\ 0.&3.4641\end{array}\right)$
and, i will need the 2×2 submatrix of eigenvalues:
$\Sigma 2 = \left(\begin{array}{cc} 84&0.\\ 0.&12\end{array}\right)$
perhaps i should point out that i have no intention of throwing away any parts of these matrices in my own work. but i need to make sure that i am following davis correctly, so i am carrying out his computations in parallel with my own.
besides, you might be trying to read davis even as i speak, another reason for me to display his work, too.
at the end of all that, all we have gotten is 2 positive eigenvalues of $X^T X$ (84 and 12), and we have their two positive square roots; we have these two pairs of numbers laid out in both 2×2 and 3×3 diagonal matrices. we have far more notation that mathematics.
so much for the eigenvalues.
next we’ll dig into the eigenvectors.