PCA / FA. Example 4 ! Davis, and almost everyone else

I would like to revisit the work we did in Davis (example 4). For one thing, I did a lot of calculations with that example, and despite the compare-and-contrast posts towards the end, I fear it may be difficult to sort out what I finally came to.

In addition, my notation has settled down a bit since then, and I would like to recast the work using my current notation.

The original (“raw”) data for example 4 was (p. 502, and columns are variables):

X_r = \left(\begin{array}{lll} 4 & 27 & 18 \\ 12 & 25 & 12 \\ 10 & 23 & 16 \\ 14 & 21 & 14\end{array}\right)
Read the rest of this entry »

Happenings March 28

There’s not a lot to say at the beginning of this weekend. I still have two drafts of posts on PCA/FA. And I have still only understood the one wavelet calculation; there are three more mysterious calculations for me to figure out. Since the one I understand leads to nice pictures, I will probably post it, even though I have provided no background information on wavelets.

I have continued to check some newsgroups for interesting questions which I can answer. One such question was posted in sci.math Thursday, March 26, about regression with a linear restriction on the coefficients.
Read the rest of this entry »

Happenings Mar 21

Well, I have a friend in town for the weekend and I’m not doing much mathematics at all. I did, however, take a day off from work before he arrived, so I’ve written drafts of two summary PCA posts. (While working on a planned “overview”, I read all 50 PCA posts; I have decided to summarize Davis again in particular.)

 

You should read mathematics backwards, from the end
to the beginning.

Salomon Bochner

 

In addition, I have figured out one of the slick wavelet calculations. Having a  calculation in front of me that clearly works makes it a lot easier to figure out what was done.
Read the rest of this entry »

Happenings Mar 14

Not a lot happened last weekend, at least nothing new really happened.

Yes, I managed to prepare and put out two posts. I feel good about that. I made a little progress on wavelets, and a little progress on category theory. That’s good too, even though I had no breakthrough in wavelets. I may, just may, have been hasty when I said the really marvelous computation – which I don’t understand – is only in one of my wavelet books; it might, just might, be related to something in two of my other books. I expect I’ll be spending time in them this afternoon.

Research is what I’m doing when I don’t know what I’m doing.

Wernher von Braun

Read the rest of this entry »

PCA/FA Answers to some Basilevsky questions

Let us look at three of the questions I asked early in February, and answer two of them.

First, what do we know? What have we done?

We assume that we have data X, with the variables in columns, as usual. In fact, we assume that the data is at least centered, and possibly standardized.

We compute the covariance matrix

c = \frac{X^T X}{N-1}\ ,

then its eigendecomposition

c = v\ \Lambda^2\ v^T\ ,

where \Lambda^2 is the diaginal matrix of eigenvalues. We define the \sqrt{\text{eigenvalue}}-weighted matrix

A = v\ \Lambda\ .

Finally, we use A as a transition matrix to define new data Z:

X^T = A\ Z^T\ .

We discovered two things. One, the matrix A is the cross covariance between Z and X:

A = \frac{X^T Z}{N-1}\ .

I find this interesting, and I suspect that it would jump off the page at me out of either Harman or Jolliffe; that is, I suspect it is written there but it didn’t register.

Two, we discovered that we could find a matrix Ar which is the cross covariance between Zc and Xs. Read the rest of this entry »

Color: odds & ends

A friend sent me an e-mail about the first color post. What started out as a few comments in reply has grown into a full-fledged post.

I think his questions boil down to:

  • how are the terms additive and subtractive related to the primaries?
  • Are the primaries unique?

Let me elaborate. Early on I’m going to mention a couple of things I think you should avoid.

Additive & Subtractive

Here are two fairly standard drawings. I believe the one on the left should be called “subtractive mixing of CMY primaries”. I believe the one on the right should be called “additive mixing of RGB primaries”.

picture-31f

We can tell at a glance which one is subtractive: it has black in the center. The other one has white in the center, so it is additive.
Read the rest of this entry »

Posted in color. 3 Comments »

Happenings Mar 7

Okay, it’s noon on Saturday. I started the morning by looking over the situation for mathematics and for the blog. Then I tried to draft this post. Then my kid interrupted and complained that he wanted to do some mathematics.

(For the blog, as for everything I write, first I write, and then I edit. First I get the ideas out, and then I dress some of them up. In the course of drafting this, I wrote “I want to sit down and do category theory right now, but I really need to put out this post. My kid will just have to wait, and he doesn’t want to…. Maybe he shouldn’t.”

So I’ve just put in some time on category theory.
Read the rest of this entry »

Posted in diary. 1 Comment »