# Estimation

Once we can identify some sort of statistical model for our data, the goal is to estimate the parameters $θ(x_{1},x_{2},…,x_{n})$ of this statistical model. This process is called estimation.

Estimation = “What is our best guess for these unknown parameters?”

We use Estimators to do estimation.

Formally:

- The Problem: $X_{1},X_{2},…,X_{n}$ is a sequence of i.i.d. r.v. with pmf $f(θ)$, where $θ$ is are the unknown true parameters
- Data: ${x_{1},x_{2},…,x_{n}}$
- The Goal: to construct $θ(x_{1},x_{2},…,x_{n})$, which is an estimate of $θ$ (true parameters)

### Naive Approach

We can test various parameters $θ$. Suppose a coin is tossed with $θ=P(Heads)$. We know that the coin is biased, where $θ=31 $ or $θ=32 $ (this is the $p$ parameter in the Binomial Distribution that we are trying to guess, $n=100$).

We run the experiment and we observe 60 heads. We can then “test” these parameters of the Binomial Distribution, where we find that:

$θ=32 ⟹P(observed data)=(60100 )(32 )_{60}(31 )_{40}=0.0307$

Therefore, you conclude that $θ=32 $ is more likely!

Now, instead of only these two parameters $θ$ that you test, what if I asked you, out of all the possible parameters $θ$ (which in this case, are ranging from $0.0$ to $1.0$, which one is the most likely?)

We can quantify the likelihood of a parameters with the Likelihood Function.

In this example, the likelihood function of a particular parameter given our observations would be $L(θ)=(60100 )(θ)_{60}(1−θ)_{40}$ We want to find the parameters $θ$ that maximizes $L(θ)$.

There exists multiple methods for Estimation: