Singular value decomposition

Suppose M is an m-by-n matrix whose entries come from the field K, which is either the field of real numbers or the field of complex numbers. A non-negative real number λ is a singular value for M if there exist non-zero vectors u in K^m and v in Kⁿ such that

Mv = λu and M^*u = λv

where M^* denotes the conjugate transpose of M. The vectors u and v are called left-singular and right-singular vectors for λ, respectively.

The singular-value decomposition theorem says that M has a factorization of the form

M = UΣ V^*

where U is an m-by-m unitary matrix over K, V is an n-by-n unitary matrix over K, and Σ is an m-by-n diagonal matrix whose diagonal entries Σ_i,i are non-negative real numbers. Such a factorization is called a singular-value decomposition of M.

In any such singular value decomposition, the diagonal entries of Σ are necessarily equal to the singular values of M.

The columns u₁,...,u_m of U are eigenvectors of MM^* and are left singular vectors of M. The columns v₁,...,v_n of V are eigenvectors of M^*M and are right singular vectors of M. Note however that different singular value decompositions of M can contain different singular vectors.

The linear transformation T: Kⁿ → K^m that takes a vector x to Mx has a particularly simple description with respect to these orthonormal bases: we have T(v_i) = d_i u_i, for i = 1,...,min(m,n), where d_i is the i-th diagonal entry of D, and T(v_i) = 0 for i > min(m,n).

The number of non-zero singular values is equal to the rank r of M. These non-zero singular values are equal to the square roots of the non-zero eigenvalues of the positive semi-definite matrix MM^*, and also equal to the square roots of the non-zero eigenvalues of M^*M.

If we focus only on these r nonzero singular values, we can construct a singular-value decomposition of the following type:

M = GDH^*

where G is an m-by-r orthonormal matrix over K, H is an n-by-r orthonormal matrix over K and D is an r-by-r diagonal matrix whose diagonal entries are positive real numbers.

The sum of the k largest singular values of M is a matrix norm, the Ky Fan k-norm of M. The Ky Fan 1-norm is just the operator norm of M as a linear operator with respect to the Euclidean norms of K^m and Kⁿ.

Add applications of singular value decomposition