Pseudoinverse

Taught to me by Kajanan. it’s not that deep.

The pseudoinverse of a matrix is a generalization of the inverse matrix, very useful for Least Squares if the matrix is not invertible / not square.

The pseudoinverse of A is defined as a matrix $A^{+}={\lim }_{\alpha \to 0}(A^{T}A+\alpha I{)}^{-1}{A}^T$

Practical algorithms compute the pseudoinverse as $A^{+}=V{D}^{+}{U}^T$

• Where $U,D$ and $V$ are the Singular Value Decomposition of $A$ ($A=UD{V}^T$)
• $D^{+}$ of a diagonal matrix $D$ is obtained by taking the reciprocal of its non-zero elements, then taking the transpose of the resulting matrix

If $A$ is $n\times m$, then $A^{+}$ is $m\times n$.

On $A$:

• When $A$ has more columns than rows, then solving a linear equation using the pseudoinverse provides one of the many possible solutions. Specifically, it provides the solution $x=A+y$ with minimal Euclidean norm $||x|{|}_2$ among all possible solutions.
• When A has more rows than columns, it is possible for there to be no solution. In this case, using the pseudoinverse gives us the x for which Ax is as close as possible to y in terms of Euclidean norm $||Ax-y|{|}_2$

Related

• SVD