Estimator

Still don’t quite understand the differences. Confidence Interval

Estimates vs Estimators

Estimates: $\hat{\theta },\overline{y},\hat{\mu }$, etc → Calculated from samples Estimators: $\tilde{\theta }$ - a RV - $\overline{y_i}$ came from $\overline{Y}$ - estimator (like $\tilde{\theta }$)

Biased vs. Unbiased Estimators

Originally heard this from David Silver’s RL course. Also wrote about these ideas in Monte-Carlo Learning. Also wrote about these ideas in Bootstrapping (Reinforcement Learning).

Aren't all estimators biased until they converge?

No, some estimators are unbiased by design. The most simple example is the estimate of the population mean, and estimate of sample mean (via Bessel’s Correction).

Learning again through STAT206.

I think when it is an unbiased estimator, then you have that $E({\hat{\sigma }}^2)={\sigma }^2$

This is the kind of proffs that they do in Research Papers, and actually we barely covered it in class, file:///Users/stevengong/My%20Drive/Waterloo/2A/STAT206/Final/STAT206/Exam%20Practice%20Full%20Solutions.pdf is a good example.

From Confidence Interval, we say that $S^2$ is an unbiased estimator of ${\sigma }^2$, but ${\hat{\sigma }}^2$ is biased.