an SVD of X ~ eigenstructures of XTX and XXT

OK, back to the drawing board. If we have a singular value decomposition of X, 

X = u\ w\ v^T

with u and v orthogonal, then

X^T X = (u\ w\ v^T)^T\ u\ w\ v^T

= v\ w^T u^T\ u\ w\ v^T

= v\ w^T w\ v^T

=: v\ D\ v^T

= v\ D\ v^{-1}

where we have defined

D = w^T w .

That is, if we have an SVD of X, then we can construct an eigendecomposition of X^T X .

(conceptually, the derivation of the SVD is nothing more than reversing that implication: given an eigendecomposition of X^T X , get an SVD for X. the challenge is to construct u.)

Note that i did not say the SVD of X, nor the eigendecomposition of X^T X . neither is unique. sometimes i may forget, so if i do refer to the SVD of a matrix or the eigendecomposition of a matrix, please forgive me. of course, “the SVD” as such means any decomposition of the form

X = u\ w\ v^T 

We can also construct an eigendecomposition of XX^T . you should expect that it uses u instead of v.

X\ X^T =(u\ w\ v^T) (u w v^T)^T

= (u\ w\ v^T)v\ w^T u^T

= u\ w\ w^T u^T

=: u\ D_2\ u^{-1} .

where we have defined

D_2 = w\ w^T .

since w is not square in general, w^T \ w \ne w\ w^T . That is, D \ne D_2 (they’re not even the same size), but both are diagonal, and both have the same nonzero entries. In fact, and we want to know this,

the nonzero entries of D or D_2 are the squares of the nonzero entries in w.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: