the SVD specializes change-of-bases

the general formula relating two matrix representations of a single linear operator L is:

B = Q^{-1}\ A\ P [change of bases]


where P and Q are invertible (hence square) matrices (of the appropriate sizes). the best reference i’ve ever seen for this is — are you ready?

the Schaum’s outline for linear algebra, by Lipschutz.

that equation is written as a transformation. We have a linear operator

L: V –>W,

and we have its matrix A WRT bases in each space V and W. then we do a change of basis P on the domain V, and a change of basis Q on the codomain W, and we find that the matrix B of L WRT the new bases is given by the change of bases equation.

we may rewrite it as a decomposition:

A = Q\ B\ P^{-1} .

we can’t replace P^{-1} by P^T because P is an arbitrary invertible matrix: we may have made any legal change-of-basis, not the special case of a rotation.
The SVD 

X = u\ w\ v^T 

is a special case of a change of basis: X and w represent the same linear operator WRT different bases. in particular, since v is orthogonal, we can replace v^T by v^{-1} :

  • X ~ A
  • w ~ B
  • u ~ Q
  • v ~ P 

So the SVD tells us that any matrix can be represented by a nearly diagonal matrix.
Why don’t we end up with a possible jordan canonical form for the SVD? or to put it another way, why does the SVD of a jordan canonical form still turn out to have nonzero terms only on the diagonal?
Because we have two matrices u and v instead of one P, to effect the decomposition.
(Of course, the eigenvector diagonalization is also a special case of the change-of-bases equation, with only one basis to be changed.)

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: