PCA / FA. Example 4 ! Davis, and almost everyone else

I would like to revisit the work we did in Davis (example 4). For one thing, I did a lot of calculations with that example, and despite the compare-and-contrast posts towards the end, I fear it may be difficult to sort out what I finally came to.

In addition, my notation has settled down a bit since then, and I would like to recast the work using my current notation.

The original (“raw”) data for example 4 was (p. 502, and columns are variables):

Happenings March 28

There’s not a lot to say at the beginning of this weekend. I still have two drafts of posts on PCA/FA. And I have still only understood the one wavelet calculation; there are three more mysterious calculations for me to figure out. Since the one I understand leads to nice pictures, I will probably post it, even though I have provided no background information on wavelets.

I have continued to check some newsgroups for interesting questions which I can answer. One such question was posted in sci.math Thursday, March 26, about regression with a linear restriction on the coefficients.

Happenings Mar 21

Well, I have a friend in town for the weekend and I’m not doing much mathematics at all. I did, however, take a day off from work before he arrived, so I’ve written drafts of two summary PCA posts. (While working on a planned “overview”, I read all 50 PCA posts; I have decided to summarize Davis again in particular.)

 

You should read mathematics backwards, from the end
to the beginning.

Salomon Bochner

 

In addition, I have figured out one of the slick wavelet calculations. Having a  calculation in front of me that clearly works makes it a lot easier to figure out what was done.

Happenings Mar 14

Not a lot happened last weekend, at least nothing new really happened.

Yes, I managed to prepare and put out two posts. I feel good about that. I made a little progress on wavelets, and a little progress on category theory. That’s good too, even though I had no breakthrough in wavelets. I may, just may, have been hasty when I said the really marvelous computation – which I don’t understand – is only in one of my wavelet books; it might, just might, be related to something in two of my other books. I expect I’ll be spending time in them this afternoon.

Research is what I’m doing when I don’t know what I’m doing.

Wernher von Braun

PCA/FA Answers to some Basilevsky questions

Let us look at three of the questions I asked early in February, and answer two of them.

First, what do we know? What have we done?

We assume that we have data X, with the variables in columns, as usual. In fact, we assume that the data is at least centered, and possibly standardized.

We compute the covariance matrix

,

then its eigendecomposition

,

where is the diaginal matrix of eigenvalues. We define the -weighted matrix

.

Finally, we use A as a transition matrix to define new data Z:

.

We discovered two things. One, the matrix A is the cross covariance between Z and X:

.

I find this interesting, and I suspect that it would jump off the page at me out of either Harman or Jolliffe; that is, I suspect it is written there but it didn’t register.

Two, we discovered that we could find a matrix Ar which is the cross covariance between Zc and Xs.

Color: odds & ends

A friend sent me an e-mail about the first color post. What started out as a few comments in reply has grown into a full-fledged post.

I think his questions boil down to:

• how are the terms additive and subtractive related to the primaries?
• Are the primaries unique?

Let me elaborate. Early on I’m going to mention a couple of things I think you should avoid.

Additive & Subtractive

Here are two fairly standard drawings. I believe the one on the left should be called “subtractive mixing of CMY primaries”. I believe the one on the right should be called “additive mixing of RGB primaries”.

We can tell at a glance which one is subtractive: it has black in the center. The other one has white in the center, so it is additive.

Happenings Mar 7

Okay, it’s noon on Saturday. I started the morning by looking over the situation for mathematics and for the blog. Then I tried to draft this post. Then my kid interrupted and complained that he wanted to do some mathematics.

(For the blog, as for everything I write, first I write, and then I edit. First I get the ideas out, and then I dress some of them up. In the course of drafting this, I wrote “I want to sit down and do category theory right now, but I really need to put out this post. My kid will just have to wait, and he doesn’t want to…. Maybe he shouldn’t.”

So I’ve just put in some time on category theory.

Books Added: More Color

The following books have been added to the bibliography.

Two of them are collections of colors. Starmer is for interior decorating, and its colors are not RGB or CMYK but rather suggestive names. I had thought I might find a specific paint maufacturer named in the book, but I didn’t. Well, if I have to, I can scan a page and let my computer figure out what colors it has on it. In any case, this book is distinguished by its marvelous form of presenting its color combinations. Most of a page is taken up by the main color, then there are two small accent colors and two smaller highlight colors.

Kuno is “just” a collection of swatches rather than combinations, except that each page has something in common, such as “sky & wind”. I find some of these colors very attractive. They all have CMYK specifications.

Gonzalez & Woods is a very recent book on digital image processing. My only general text was ancient, so I followed a newsgroup recommendation and bought this. (I had bought a specialized text last year just because someone needed help on a wavelet problem in it!) It turns out that they have the HSI color model, the one with trigonometry.

Finally, Rossotti is a short paperback introduction to color in general, but it gets just enough into the details to be interesting. For example, although I don’t really, really care how photographic emulsions combine to give us color photography, her discussion was detailed enough to make sense, general enough to not drive me away.