The singular value decomposition (SVD) is marvelous. It says that any real matrix X can be decomposed as
[singular value decomposition]
where u and v are orthogonal (i.e. rotations) and w is as close to diagonal as possible.To be more precise,
where in fact is not only a diagonal matrix but it has only positive elements. Finally, we order the elements of w from largest to smallest(don’t sweat it if some are equal).
(the theorem is true for any complex matrix, provided the transpose is replaced by the conjugate transpose, and u and v are unitary instead of orthogonal. i just want to stick with the transpose. Or you could simply interpret as the conjugate transpose of v in everything i do.)
My main reference for this is Stewart: short and sweet, with some history and the derivation. Strang has a gentler discussion, but no derivation.