PCA / FA Malinowski Summary

Malinowski’s work is considerably different from everything else we’ve seen before.

First of all, he expects that in most cases one will neither standardize nor even center the data X. We can do his computations as an SVD of X, or an eigendecomposition of or of – but because the data isn’t even centered, and are not remotely covariance matrices. For this reason, I assert that preprocessing is a separate issue.

PCA / FA Malinowski: Example 5. Simplified Target Testing

introduction

For a good reason which I have not yet discussed, Malinowski wants to find x vectors which are close to vectors. (His x and are usually written y and for a least-squares fit). He finds a possible x and tests it to see if xhat is close. He recommended computing an intermediate t vector which was the for his least-squares fit to x.

Since he seems to care about t only when is close to x, and since is incredibly easy to compute directly, I prefer to delay the computation of t. Find t after we’ve found a good x.

It will also turn out that he wants a collection of t vectors in order to pick a nicer basis than u or u1. And I’m not going to follow him there, because all of that is what practitioners call “non-orthogonal rotations”. (That strikes me as an oxymoron.) It’s what Harman spends most of his book doing, and that’s where I’ll look if I ever I want to. It’s important, but I’m not going to look at it this time around.

Anyway, we factored the data matrix

PCA / FA malinowski: example 5. target testing

Recall that we computed the SVD of this matrix:

and we found that the w matrix was

Because w has only two nonzero entries, we know that X is of rank 2. Its three columns only span a 2D space.

Given a column of data x (a variable, in this example, of length 5), Malinowski wants to know if it is in that 2D space. As he puts it, “if the suspected test vector [x] is a real factor, then the regeneration will be successful.” He gives us a formula for computing t; by a successful regeneration, he means that is close to x.

PCA / FA Malinowski: example 5.

(June 10: i have made 4 edits, all cosmetic. you may search on “edit:”)

Malinowski (edit: “Factor Analysis in Chemistry”, 3rd ed.) does a lot of things differently from what we’ve seen. Fortunately, his model is simple enough, although his notation is… different. His model is

X = R C,

and he calls R and C the row and column matrices respectively. He wants X to have more rows than columns, so he transposes if necessary; then he chooses C to have more columns than rows, and R will have more rows than columns. For starters, then, his X matrix looks like the usual design matrix for regression. (Incidentally, he didn’t call it X.)

He chooses , from the cut-down SVD. That is, I write the SVD of X as

,

where u and v are orthogonal and w is the same shape as X. But we know from the derivation and our experience with Davis that we may also write

,

where is square, diagonal, and invertible (it is a cut-down w), and and are the submatrices of u and v which are conformable with . (We’ll see all this shortly.) We have dropped the parts of u, w, and v which are not required for reproducing X. (I remind you that what we’ve lost is the orthogonality of the matrices u and v.)