Moment-Generating Function

Moment-Generating Function (Definition)

The moment generating function (mgf) of a r.v. $X$ is:
$M_X(t)=E(e^{tx})$

• $E(e^{tx})$ is a function of $t$ ($t$ is a dummy variable)
• $E(e^{tx})$ needs to be finite on an open interval containing $0$, otherwise we say the mgf doesn’t exist

Ex: Find the mgf for Bernoulli Distribution: $M_X(t)=E(e^{tx})={\sum }_{x\in X}{e}^{tx}(f(x))=e^0(1-p)+e^{t}p$

Ex: Find the mgf of a Uniform Distribution: $M_U(t)=E(e^{tu})={\int }_a^b{e}^{tu}\cdot \frac{1}{b-a}du=\frac{e^{tb}-e^{ta}}{t(b-a)}$

• 1st moment: $E(x)=\mu$
• 2nd moment: $E(x^2)$ → ${\sigma }^2=Var(x)$, Variance
• 3rd moment: $E(x^3)$ → Skewness
• 4th moment: $E(x^4)$ → Kurtosis (how “fat the tail is”)

Why is the mgf important? Because we have the following results.

Result 1

We can calculate the $n$-th moment of a r.v. $X$ by evaluating the $n$-th derivative of the mgf at $t=0$

MGFs allow us to replace “messy” integration with “cleaner” derivatives.

Result 2: The mgf determines the distribution

If two RVs have the same mgf, then they have the same distribution.

Result 3: The mgf of indenpendent distributions

If r.v.s $X$ and $Y$ are independent, then $M_{X+Y}(t)=M_X(t)\cdot M_Y(t)$

This is useful because we are often interested in sums and averages in Modelling.

Example: Find the mgf of a Binomial Distribution, i.e. find the mgf of $X$, where $X\sim Bin(n,p)$. $M_X(t)=E[e^{tx}]=e^{t\cdot 0}\left(\genfrac{}{}{0ex}{}{n}{0}\right)p^0(1-p{)}^n+e^{t-1}(\genfrac{}{}{0ex}{}{n}{1})p^1(1-p{)}^{n-1}+\cdots$ We know that $X=X_1+\cdots +X_n$ where $X_i\sim Ber(p)$

Using Result 3, we get $M_X(t)=M_{X_1}(t)\cdot M_{X_2}(t)\dots M_{X_n}(t)$ $⟹M_X^{\prime }(0)=np\leftarrow E(X)$

TODO: COMPLETE THE PROOF HRE M_1_X WHAT IS happening here????