LogSumExp

LogSumExp is defined as the logarithm of the sum of the exponentials of the arguments: $\mathrm{LSE}(x_1,\dots ,x_n)=\log \left(\exp (x_1)+\cdots +\exp (x_n)\right)$

When is this used? Ex: $log(e^{1000}+e^{999})$

To stabilize the computation, we factor out the largest exponent:

$\log \left({\sum }_{i=1}^n{e}^{x_i}\right)=M+\log \left({\sum }_{i=1}^n{e}^{x_i-M}\right)$ Where: $M=max(x_1,x_2,...,x_n)$

Derivation: Just take out the largest exponent, you can compute it by hand https://chatgpt.com/share/67a00c3c-16d8-8002-beab-33d689d02915