## Why not orthogonal?

Consider the diagonalization of a matrix:

$D = P^{-1}\ A\ P\$,

or

$A = P\ D\ P^{-1}\$.

A friend asked me why I couldn’t always choose P to be an orthogonal (more generally, a unitary) matrix. Actually, it was more like: remind me why I can’t do that. I fumbled for the answer. Sad dog.

We recall that an orthogonal matrix Q is real, square and satisfies

$Q^T\ Q = I\$,

where $Q^T$ is the transpose of Q. That is to say, the inverse of Q is its transpose: $Q^T = Q^{-1}\$. If our eigenvector matrix P is orthogonal, we may replace

$A = P\ D\ P^{-1}\$.

by

$A = P\ D\ P^{T}\$.

A unitary matrix is complex (possibly real), square and satisfies

$Q^{\dagger}\ Q = I\$,

where$Q^{\dagger}$ is the conjugate transpose of Q. That is to say, the inverse is the conjugate transpose: $Q^{\dagger} = Q^{-1}\$.

Finally, we recall that a matrix is normal if it commutes with its complex conjugate:

$A^{\dagger}\ A = A\ A^{\dagger}\$.

Now we have to go all the way back to the schur’s lemma post here and recall that

1. Any matrix A can be brought to upper triangular form by some unitary matrix.
2. A matrix A can be diagonalized by some unitary matrix if and only if A is normal.

That is to say, our guaranteed upper triangular matrix is in fact diagonal if and only if A is normal.

$D = P^{-1}\ A\ P\$,
I suppose I should remind us all that the matrices $X\ X^T$ and $X^T\ X$ (cf. covariance and correlation matrices, R-mode and Q-mode) are normal, so that if we diagonalize them, our eigenvector matrices are orthogonal.
As for the SVD, $X = u\ w\ v^T\$, we are using two matrices u and v, not one matrix P, and u and v are guaranteed unitary.