A square matrix may have directions it scales without turning: \(A\mathbf{x}=\lambda\mathbf{x}\). These are eigenvectors; the eigenvalues \(\lambda\) give the scale, including a reversal when negative.
1 Eigendecomposition
When the eigenvectors form a basis, collect them as columns of \(V\). Then \(A=V\Lambda V^{-1}\) describes three steps:
- \(V^{-1}\) expresses the input as amounts along the eigenvectors.
- \(\Lambda\) multiplies each amount by its eigenvalue.
- \(V\) combines the scaled eigenvectors in the original coordinates.
\(V^{-1}\) inverts the whole matrix of eigenvectors, not individual vectors. It changes coordinates; it does not undo \(A\). For symmetric \(A\), the eigenvectors can be perpendicular and \(V^{-1}=V^T\).
A rectangular matrix maps between spaces of different dimensions and has no eigenvectors in this sense. Some square matrices lack a full real eigenvector basis.
2 Singular Value Decomposition
Every real matrix instead has \(A=U\Sigma V^T\):
- \(V^T\) measures the input along the right singular vectors.
- \(\Sigma\) scales those amounts by nonnegative singular values; zeros collapse directions.
- \(U\) combines the scaled amounts along the left singular vectors to form the output.
The two sets of singular vectors give perpendicular input and output axes, even when those spaces have different dimensions. Singular values show which input directions have the strongest and weakest effects.
Eigenvectors stay on their own lines; singular vectors map one orthogonal basis to another.
Axes. Stretch. Rotate. Grids. Map. Scale. Compress. Compare. Choose. Wisely.