## the SVD: nonzero singular values?

i should babble something about singular values and $\Sigma$. the issue is whether “nonzero singular values” is redundant because singular values are defined to be nonzero.
For technical purposes – the proof we just did! – it is extremely useful, even essential, to let $\Sigma$ hold precisely the nonzero elements; we write the decomposition of w into

$w=\left(\begin{array}{cc}\Sigma&O\\O&O\end{array}\right)$

so that we can invert the diagonal submatrix  $\Sigma$.

in practice, however, it is useful and conceptually convenient (for me at least) to imagine that the diagonal of $\Sigma$ is extended to whichever boundary it hits first. in practice, then, w is of one of the following forms:

$\left(\begin{array}{c}\Sigma\\O\end{array}\right)$  or  $\left(\begin{array}{cc}\Sigma&O\end{array}\right)$

then we can say

all the entries of $\Sigma$ are nonzero iff w is of full rank.

we can’t phrase it that way if $\Sigma$ is automatically made smaller to contain only nonzero entries.

illustration 1

$w=\left(\begin{array}{ccc}2&O&O\\O&1&O\\O&O&O\\O&O&O\end{array}\right)$

Technically $\Sigma$ is the 2×2 diagonal submatrix

$\left(\begin{array}{cc}2&O\\O&1\end{array}\right)$

and some people call the diagonal entries (2 and 1) of $\Sigma$  the singular values of our original matrix X. But conceptually i am interested in the 3×3 submatrix

$\left(\begin{array}{ccc}2&O&O\\O&1&O\\O&O&O\end{array}\right)$

which extends $\Sigma$ to as large a square submatrix as possible. i would say that our original 4×3 matrix w has 3 singular values, one of which is zero.

However i phrase it, this is a major observation: that there are only 2 nonzero diagonal elements (“2 nonzero singular values”) tells us that our matrix w is of rank 2 instead of rank 3. although the w matrix maps 3-space to 4-space, we know that its range can be 3-space at most. Its SVD tells us that the matrix is, in fact, a map from 4-space to 2-space. It’s image is only a plane.

matrix rank is one of the major applications of the SVD.

i do not know if the definition of singular values is standardized. i find it useful to distinguish the technical definition of $\Sigma$ in which all elements of the diagonal are nonzero, from the operational use, where we redefine  $\Sigma$ so that it may comtain some diagonal zeroes. we modify our definition to suit our needs.

or maybe we simply understand that the nonzero entries which define  $\Sigma$ do not exhaust the singular values of X.

the issue is whether “nonzero singular values” is redundant because singular values can only be nonzero. In discussions, i prefer to permit zero singular values.

### One Response to “the SVD: nonzero singular values?”

1. The Domain Says:

Good post. I certainly love this site. Thanks!