the general formula relating two matrix representations of a single linear operator L is:
[change of bases]
where P and Q are invertible (hence square) matrices (of the appropriate sizes). the best reference i’ve ever seen for this is — are you ready?
the Schaum’s outline for linear algebra, by Lipschutz.
that equation is written as a transformation. We have a linear operator
L: V –>W,
and we have its matrix A WRT bases in each space V and W. then we do a change of basis P on the domain V, and a change of basis Q on the codomain W, and we find that the matrix B of L WRT the new bases is given by the change of bases equation.
we may rewrite it as a decomposition:
we can’t replace by because P is an arbitrary invertible matrix: we may have made any legal change-of-basis, not the special case of a rotation.
is a special case of a change of basis: X and w represent the same linear operator WRT different bases. in particular, since v is orthogonal, we can replace by :
- X ~ A
- w ~ B
- u ~ Q
- v ~ P
So the SVD tells us that any matrix can be represented by a nearly diagonal matrix.
Why don’t we end up with a possible jordan canonical form for the SVD? or to put it another way, why does the SVD of a jordan canonical form still turn out to have nonzero terms only on the diagonal?
Because we have two matrices u and v instead of one P, to effect the decomposition.
(Of course, the eigenvector diagonalization is also a special case of the change-of-bases equation, with only one basis to be changed.)