Matrix Decomposition

Eigendecomposition

Eigendecomposition is a method to decompose a matrix into a set of Eigenvalues and Eigenvectors.

Remember that if $v$ is an eigenvector of $A$, then $Av=\lambda v$

Suppose matrix $A$ has $n$ linearly independent eigenvectors $\{\overrightarrow{v_1},\dots ,\overrightarrow{v_n}\}$ with corresponding eigenvalues $\{{\lambda }_1,\dots ,{\lambda }_n\}$.

We can construct the matrix $V$ with one eigenvector per column: $V=[\overrightarrow{v_1},\dots ,\overrightarrow{v_n}]$.

The eigendecomposition is given by $A=Vdiag(\lambda )V^{-1}$

Derivation

When does this eigendecomposition exist? And why does it work? Let’s first form the matrix $V$. Notice that by definition of eigenvalues and eigenvectors, You will have: $AV=Vdiag(\lambda )$ So multiplying by the inverse, we get: $A=Vdiag(\lambda )V^{-1}$

If $A$ is a real symmetric matrix, then we can decompose it into $A=QD{Q}^T$ where

• $Q$ is an orthogonal matrix composed of eigenvectors of $A$
• $D$ is a diagonal matrix composed of the eigenvalues

Two questions:

1. Why does this always work? For symmetric matrices, the eigenvectors are guaranteed to be orthogonal. So there always exists a eigendecomposition.
2. Why can we just take the transpose instead of the inverse?? Because $Q$ is an Orthogonal Matrix, so by definition!

Note that eigendecomposition is not guaranteed to be unique.

The matrix is Singular if and only if any of the eigenvalues are zero.

Eigendecomposition only works on square matrices

Because eigenvalues are only defined for square matrices.

The eigendecomposition of a real symmetric matrix can also be used to optimize quadratic expressions of the form $f(x)=x^{T}Ax$ subject to $||x|{|}^2=1$.

Related

• SVD (which generalizes the eigendecomposition)