i should babble something about singular values and . the issue is whether “nonzero singular values” is redundant because singular values are defined to be nonzero.
For technical purposes – the proof we just did! – it is extremely useful, even essential, to let hold precisely the nonzero elements; we write the decomposition of w into
so that we can invert the diagonal submatrix .
in practice, however, it is useful and conceptually convenient (for me at least) to imagine that the diagonal of is extended to whichever boundary it hits first. in practice, then, w is of one of the following forms:
then we can say
all the entries of are nonzero iff w is of full rank.
we can’t phrase it that way if is automatically made smaller to contain only nonzero entries.
Technically is the 2×2 diagonal submatrix
and some people call the diagonal entries (2 and 1) of the singular values of our original matrix X. But conceptually i am interested in the 3×3 submatrix
which extends to as large a square submatrix as possible. i would say that our original 4×3 matrix w has 3 singular values, one of which is zero.
However i phrase it, this is a major observation: that there are only 2 nonzero diagonal elements (“2 nonzero singular values”) tells us that our matrix w is of rank 2 instead of rank 3. although the w matrix maps 3-space to 4-space, we know that its range can be 3-space at most. Its SVD tells us that the matrix is, in fact, a map from 4-space to 2-space. It’s image is only a plane.
matrix rank is one of the major applications of the SVD.
i do not know if the definition of singular values is standardized. i find it useful to distinguish the technical definition of in which all elements of the diagonal are nonzero, from the operational use, where we redefine so that it may comtain some diagonal zeroes. we modify our definition to suit our needs.
or maybe we simply understand that the nonzero entries which define do not exhaust the singular values of X.
the issue is whether “nonzero singular values” is redundant because singular values can only be nonzero. In discussions, i prefer to permit zero singular values.