PCA / FA Example 4: Davis. review (1)

It’s taking me a while to summarize Davis, partly because it isn’t just a summary. There are several interesting details to discuss, and then I need to ask: what did we see Harman and jolliffe do?

While I work on My Answers, let me show you the outline of interesting details, and barely refresh your memory of Harman & jolliffe.

In addition, I will show you one answer that blew me away. Maybe it shouldn’t have, but it did.

test input for biblio

This post is a test of a new way of handling the bibliography. Sheesh!

For one thing, I’m using the Firefox browser instead of Safari: carriage returns no longer disappear.

For another, I’ve discovered that I can put some HTML tags in my source without having to put all of them in: the wordpress editor keeps mine and adds whatever it needs. In particular, I can make the bold tags around the book titles a part of the master copy, instead of editing it one title at a time.

I still have to go edit my current complete bibliography to match this format, and let me sort it automatically. As usual, trying to solve one problem, I saw several useful tricks.

That is, these books have not yet been added to the bibliographies page.

Oh, of course they have been by now.

Here be dragons – their care and feeding: i.e. “technical remarks”

older posts
if you click on the “math PCA” category, you get a page on which the earliest post is dated Feb 11; it is one of the Jolliffe posts, but not the first one. where are the rest? there is a link at the bottom of that page, “older posts”, and it will give you all the rest of the posts in that category.

blech

if you see any of those yellow and red signs (excluding that one, of course), please try again in a little while. the chances are very high that wordpress is being flaky. “it’s not my fault.”

i do make mistakes, and there are typos in these posts, but that image is real hard to miss; it’s exactly what i see when i preview an equation that isn’t acceptable latex to wordpress. i promise you: there aren’t any of those left when i publish a post. that much i do get right.

please keep coming back despite some technical glitches.

searching
on a mac, apple-F opens up a search box. it is limited to the current page, but you can change the page. on windows, control-F opens a search box. the exact form and location depends on the browser.

bibliography
as i said last weekend, the wordpress editor now eats carriage returns, and i am terrified of editing the bibliography. i’ll find a way, but until i do, note books that are referenced in posts.

Happenings – 24 March (3) Quantum Mechanics

this is the third post of its kind. as i said in the first happenings post, i had three breakthroughs last weekend: in PCA, controls, and quantum mechanics. the controls was in post (2), the PCA is coming; this is the quantum mechanics.

at the beginning of what is now last weekend, i had a breakthru in quantum mechanics. i found a simple example of one of the things i was trying to calculate. this came from McMahon’s “Quantum Mechanics Demystified”. he is also the author of “Relativity Demystified”. both books have some really excellent examples in them, things not usually found in introductory books. the bad news is, his quantum mechanics (henceforth QM) text is marred by an awful lot of typos. the good news is, it’s cheap.

i can summarize my overall reaction as follows. i just checked for a 2nd edition which might have fixed the typos. there isn’t one. but he does have “quantum field theory demystified” and i’m going to order it, just because he’s the author. i’ll take my chances with typos. i’m expecting to see some informative examples anyway.

Happenings – 24 March (2) Control Theory

My general question in control theory was: should I work thru some ancient classical design problems, as they do, with root-locus, nyquist, Nichols, and bode plots? As a minor issue, I would need to select examples from among a few books. But the design methodology appeared to be: use frequency domain analysis to estimate control system parameter values which would lead to a desirable time-domain response of the system.

It occurred to me that my computer and Mathematica® are powerful enough to show the time-domain response in real time.

PCA / FA Example 4: Davis. R-mode & Q-mode via the SVD.

let’s finally do this using the SVD. i need to do one terrible thing: where davis writes U and V, we need v and u, resp. (look, i couldn’t reliably keep translating davis’ equations in my head, so i had to use his notation; by the same token, i can’t reliably translate the SVD over and over again. thank god he used UC, upper case.)

if the correspondence had been u ~ U and v ~ V, the translation would have been trivial. unfortunately, the correspondence is

u ~ V

v ~ U.

(from the SVD posts, you recall that given we conclude that v is an eigenvector matrix for , and u is an eigenvector matrix for . and, just as important, the nonzero values of w are the nonzero .)

Happenings – 24 March

Last week’s happenings post was written Saturday. Later that day I had a great idea for controls. At the beginning of this weekend I found a great example in quantum mechanics; and yesterday afternoon I had a breakthrough in PCA / FA. I think I will describe the first two in separate posts; the PCA will show up soon, in sequence. (The quantum mechanics and controls will not be in one big “happenings” post. my primary reason for keeping posts smaller rather than larger is so that the latex translation is of manageable size.)

Last Monday I went to the annual Oppenheimer Lecture at Cal. it was about quantum mechanics, “spooky actions at a distance”. The speaker’s overheads are available on his webpage http://people.ccmr.cornell.edu/~mermin/homepage/spooky-berk.pdf

PCA / FA example 4: Davis. R-mode & Q-mode related

for this section, i need – i want – to match davis’ signs and sizes. this is just temporary.

instead of my …

i need to change the sign of the 2nd column and drop the 3rd:

instead of my …

i need to change the sign of the 1st column and drop the last two:

note

 there were two posts today. the first was a straight-shot computation of R-mode; feel free to ignore it if you’re comfortable with that material.

PCA / FA Example 4: Davis. Q-mode.

that was R-mode. Q-mode is similar. in fact, it’s more than similar, but we’ll get to that. the starting point is to form instead of :

x

= .

where was 3×3 because we had 3 variables, is 4×4 because we have 4 observations. clearly, if we have 100 observations, will be 100×100, etc. that could be hard to work with numerically; it would very likely be overwhelming conceptually. not sure about you, but i don’t want to look for structure in a 100×100 matrix!

guess what? a light dawns. i see why one might not want to display all of the V matrix. and for large enough data sets, one might not compute it. orthogonal be damned, if it’s too damned big!

there is a method to their madness, at least computationally. nevertheless, conceptually i will continue to use the SVD with u and v square and orthogonal.

guess what? another light dawns. i wonder if one might sometimes choose to compute only an eigendecomposition of instead of an SVD of X; we would get only the smaller eigenvector matrix, without the larger u and w matrices (in ). this could be a very useful dodge.