## an SVD of X ~ eigenstructures of XTX and XXT

OK, back to the drawing board. If we have a singular value decomposition of X,

$X = u\ w\ v^T$

with u and v orthogonal, then

$X^T X = (u\ w\ v^T)^T\ u\ w\ v^T$

$= v\ w^T u^T\ u\ w\ v^T$

$= v\ w^T w\ v^T$

$=: v\ D\ v^T$

$= v\ D\ v^{-1}$

where we have defined

$D = w^T w$ .

That is, if we have an SVD of X, then we can construct an eigendecomposition of $X^T X$ .

(conceptually, the derivation of the SVD is nothing more than reversing that implication: given an eigendecomposition of $X^T X$ , get an SVD for X. the challenge is to construct u.)

Note that i did not say the SVD of X, nor the eigendecomposition of $X^T X$ . neither is unique. sometimes i may forget, so if i do refer to the SVD of a matrix or the eigendecomposition of a matrix, please forgive me. of course, “the SVD” as such means any decomposition of the form

$X = u\ w\ v^T$

We can also construct an eigendecomposition of $XX^T$ . you should expect that it uses u instead of v.

$X\ X^T =(u\ w\ v^T) (u w v^T)^T$

$= (u\ w\ v^T)v\ w^T u^T$

$= u\ w\ w^T u^T$

$=: u\ D_2\ u^{-1}$ .

where we have defined

$D_2 = w\ w^T$ .

since w is not square in general, $w^T \ w \ne w\ w^T$ . That is, $D \ne D_2$ (they’re not even the same size), but both are diagonal, and both have the same nonzero entries. In fact, and we want to know this,

the nonzero entries of D or $D_2$ are the squares of the nonzero entries in w.