the SVD: nonzero singular values?

i should babble something about singular values and \Sigma . the issue is whether “nonzero singular values” is redundant because singular values are defined to be nonzero.
For technical purposes – the proof we just did! – it is extremely useful, even essential, to let \Sigma hold precisely the nonzero elements; we write the decomposition of w into

w=\left(\begin{array}{cc}\Sigma&O\\O&O\end{array}\right)  

so that we can invert the diagonal submatrix  \Sigma .

in practice, however, it is useful and conceptually convenient (for me at least) to imagine that the diagonal of \Sigma  is extended to whichever boundary it hits first. in practice, then, w is of one of the following forms:     

\left(\begin{array}{c}\Sigma\\O\end{array}\right)  or  \left(\begin{array}{cc}\Sigma&O\end{array}\right)  

then we can say

all the entries of \Sigma  are nonzero iff w is of full rank.

we can’t phrase it that way if \Sigma  is automatically made smaller to contain only nonzero entries.

illustration 1

w=\left(\begin{array}{ccc}2&O&O\\O&1&O\\O&O&O\\O&O&O\end{array}\right)

Technically \Sigma  is the 2×2 diagonal submatrix

\left(\begin{array}{cc}2&O\\O&1\end{array}\right)

and some people call the diagonal entries (2 and 1) of \Sigma  the singular values of our original matrix X. But conceptually i am interested in the 3×3 submatrix

\left(\begin{array}{ccc}2&O&O\\O&1&O\\O&O&O\end{array}\right)

which extends \Sigma  to as large a square submatrix as possible. i would say that our original 4×3 matrix w has 3 singular values, one of which is zero. 

However i phrase it, this is a major observation: that there are only 2 nonzero diagonal elements (“2 nonzero singular values”) tells us that our matrix w is of rank 2 instead of rank 3. although the w matrix maps 3-space to 4-space, we know that its range can be 3-space at most. Its SVD tells us that the matrix is, in fact, a map from 4-space to 2-space. It’s image is only a plane.

matrix rank is one of the major applications of the SVD. 

i do not know if the definition of singular values is standardized. i find it useful to distinguish the technical definition of \Sigma in which all elements of the diagonal are nonzero, from the operational use, where we redefine  \Sigma so that it may comtain some diagonal zeroes. we modify our definition to suit our needs.

or maybe we simply understand that the nonzero entries which define  \Sigma do not exhaust the singular values of X.

the issue is whether “nonzero singular values” is redundant because singular values can only be nonzero. In discussions, i prefer to permit zero singular values.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: