the Singular Value Decomposition (SVD)

The singular value decomposition (SVD) is marvelous. It says that any real matrix X can be decomposed as

X = u\ w\ v^T [singular value decomposition]

where u and v are orthogonal (i.e. rotations) and w is as close to diagonal as possible.To be more precise,

w = \left(\begin{array}{cc}\Sigma&O\\O&O\end{array}\right)

where in fact \Sigma is not only a diagonal matrix but it has only positive elements. Finally, we order the elements of w from largest to smallest(don’t sweat it if some are equal).
(the theorem is true for any complex matrix, provided the transpose is replaced by the conjugate transpose, and u and v are unitary instead of orthogonal. i just want to stick with the transpose. Or you could simply interpret v^T as the conjugate transpose of v in everything i do.)
My main reference for this is Stewart: short and sweet, with some history and the derivation. Strang has a gentler discussion, but no derivation.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: