Confidence Interval

A confidence interval (CI) is a range of estimates for an unknown parameter.

The goal is to fix a probability, and then find an interval where $100\ast p$% of such intervals conform to the parameter of interest.

Isn’t that what we did for the Likelihood Interval??

$\theta$ - unknown parameter (a constant i.e $mu,\sigma ,{\sigma }^2,\lambda$, etc), $\hat{\theta }$ - estimate to $\theta$ → a # calculated based on data - $\hat{\mu },\hat{\lambda },\hat{\theta }$, etc. $\tilde{\theta }$ - a RV that corresponds to $\hat{\theta }$ - RV of which $\hat{\theta }$ is an outcome

Estimator

Confidence Interval (Definition)

A $100\ast p$% Confidence Interval for $\theta$ is an estimate of the interval
$[L,U]$ for RVs, $L(Y_1,...,Y_n)$ and $U(Y_1,...,Y_n)$, such that $P(L(Y_1,\dots ,Y_n)<\theta <U(Y_1,\dots ,Y_n))=p$

The interval $[l(y_1,\dots ,y_n),u(y_1,\dots ,y_n)]=100\ast p\%$

Interpretation of Confidence Intervals

Suppose we have a 95% CI: $\hat{\theta }\pm a$, where $\hat{\theta }=20$, $a=10$ $20\pm 10⟹[10,30]\leftarrow \text{over 95 percent CI}$ does NOT mean there is a 95% chance that the the $[10,30]$ interval contains the parameter. Instead, 95% of constructed CIs fall around (contain) the mean.

It means that 95% of intervals will contain $\theta$

Also see Interval Estimation.

Example

Suppose that we have $Y_1,\dots Y_{25}\sim N(\mu ,144)$ and we have collected $\{y_1,...,y_{25}\}$ and $\overline{y}$.

Construct the 95% Confidence Interval for $\mu$.

We will do this in 3 steps:

1. Construct the Pivotal Quantity
2. Using the Pivotal Distribution, construct the Coverage Interval
3. Estimate the coverage interval using your data

Step 1: Construct the Pivotal Quantity

The Pivotal quantity is given by $Z=\frac{\overline{Y}-\mu }{\frac{\sigma }{\sqrt{n}}}$ From the above, we known ${\sigma }^2=144$, and $n=25$, therefore $\frac{\sigma }{\sqrt{n}}=\frac{12}{5}$ $\overline{Y}\sim N\left(\mu ,\frac{144}{25}\right)⟹\frac{\overline{Y}-\mu }{\frac{12}{5}}=Z\sim N(0,1)$

Step 2: Using the Pivotal distribution, construct the Coverage interval

Since we want the 95% confidence interval, we look at the $N(0,1)$ quantiles in the Z-Table for and find a value of of $1.96$ for $p=0.975$. We do this because we want the centered around the mean.

Make sure to draw your Normal Distribution, writing down your numbers to really convince yourself.

• The general trick is just say, they ask you for the Y% interval. You should take the values $[-(Y+\frac{1-Y}{2}),(Y+\frac{1-Y}{2})]$ as your bounds. Therefore, we get that $P(-1.96<Z<1.96)=0.95$ We then isolate for $\mu$: $⟹P(\overline{Y}-1.96\times \frac{12}{5}<\mu <\overline{Y}+1.96\times \frac{12}{5})$

Step 3: Estimate the Coverage Interval using your data

Best estimate of $\overline{Y}$ is $\overline{y}$ $CI=\overline{y}\pm a\frac{\sigma }{\sqrt{n}}$

• where $\frac{\sigma }{\sqrt{n}}$ is the Standard Error

So we obtain our final answer after fitting int he data:
$CI=75\pm 1.96(\frac{12}{5})$

Normal Confidence Intervals

I guess this is a summary of what we just did above? $Y_1,\dots ,Y_n\sim N(\mu ,{\sigma }_0^2)$

• $\mu$ and ${\sigma }_0^2$ are known parameters here $⟹\overline{Y}\sim N(\mu ,\frac{{\sigma }_0^2}{\sqrt{n}})$ $P(-z^{\ast }<\frac{\overline{y}-\mu }{\frac{\sigma }{\sqrt{n}}}<z^{\ast })=p$ $\overline{y}\pm z^{\ast }\frac{{\sigma }_0}{\sqrt{n}}$
• The general format the the above is Mean $\pm$ critical value * Standard Error

Unknown $\mu$ and ${\sigma }^2$

However, sometimes, $\mu$ and ${\sigma }^2$ are unknown to us. In these cases, we can still find the Confidence Interval for both parameters.

If $Y_1,\dots ,Y_n$ are i.i.d. $N(\mu ,{\sigma }^2)$, $\overline{Y}=\frac{1}{n}{\sum }_{i=1}^n{Y}_i$ and $S^2=\frac{1}{n-1}{\sum }_{i=1}^n(Y_i-\overline{Y}{)}^2$ then

We can write this in terms of the Student’s t-Distribution to estimate the Student’s t-Distribution to estimate the Mean: $\frac{\overline{Y}-\mu }{\frac{S}{\sqrt{n}}}\sim T(n-1)$ We can also write this in terms of the Chi-Squared Distribution to estimate the Chi-Squared Distribution to estimate the Variance: $\frac{(n-1)S^2}{{\sigma }^2}\sim {\chi }^2(n-1)$

The above notation is confusing, so just some clarification:

• $\overline{Y}$ and $S$ are RVs, $\overline{y}$ and $s$ and estimates
• The estimate of $S^2$ is $s^2$. We say that $S^2$ is an Unbiased Estimator of ${\sigma }^2$ but ${\hat{\sigma }}^2$ is biased.
  • Basically, $s^2$ is what we actually see in our data, and to get it to be unbiased, we reduce it by one degree of freedom

CI for $\mu$

$Y_1,\dots ,Y_n\sim N(\mu ,{\sigma }^2),\mu ,\sigma$ are unknown.

A sample of 16 students were taken and it was calculated that $\overline{y}=75$ and $s^2=100$. Find a 90% CI for $\mu$.

Step 1: Construct the Pivotal Quantity We have that $\frac{\overline{Y}-\mu }{\frac{10}{\sqrt{16}}}\sim T(16-1)$ Coverage Interval: 90% CI, and 15 DOF, looking at our t-table, we actually need to use the value that is for p=0.95 and $dof=15$.

$P(-1.75<T(15)<1.75)=0.9$ $P\left(\overline{Y}-1.75\frac{10}{\sqrt{16}}<\mu <\overline{y}+1.75\ast \frac{10}{\sqrt{16}}\right)=0.9$ $\overline{y}\pm z^{\ast }\frac{{\sigma }_0}{\sqrt{n}}$ Confidence Interval: $\overline{y}\pm 1.75\frac{10}{\sqrt{16}}=75\pm 4.375$

CI for ${\sigma }^2$

Pivotal Quantity: If $Y_1,\dots ,Y_n$ are i.i.d. $N(\mu ,{\sigma }^2)$, $\overline{Y}=\frac{1}{n}{\sum }_{i=1}^n{Y}_i$ and $S^2=\frac{1}{n-1}{\sum }_{i=1}^n(Y_i-\overline{Y})$ then $\frac{(n-1)S^2}{{\sigma }^2}\sim {\chi }^2(n-1)$

Coverage Interval: $[\frac{(n-1)S^2}{a},\frac{(n-1)S^2}{b}]$ Confidence Interval: $[\frac{(n-1)S^2}{a},\frac{(n-1)S^2}{b}]$

Binomial CI

If $Y\sim Bin(n,\theta )$, then we have following results:

Pivotal Quantity $Y\sim N(n\theta ,n\theta (1-\theta ))$ Pivotal Distribution $\frac{Y-n\theta }{\sqrt{n\theta (1-\theta )}}=Z\sim N(0,1)$ Coverage Interval $\tilde{\theta }\pm z^{\ast }\sqrt{\frac{\tilde{\theta }(1-\tilde{\theta })}{n}}$ Confidence Interval $\tilde{\theta }\pm z^{\ast }\sqrt{\frac{\tilde{\theta }(1-\tilde{\theta })}{n}}$

Poisson CI

If $Y_1,\dots ,Y_n\sim Poi(\theta )$, then we have following results: Pivotal Quantity $\overline{Y}\sim N(\theta ,\frac{\theta }{n})$ Pivotal Distribution $\frac{\overline{Y}-\theta }{\sqrt{\frac{\overline{Y}}{n}}}=Z\sim N(0,1)$ Coverage Interval $\overline{Y}\pm z^{\ast }\sqrt{\frac{\overline{Y}}{n}}$ Confidence Interval $\overline{y}\pm z^{\ast }\sqrt{\frac{\overline{y}}{n}}$