Bayes Rule

Posterior Probability

Posterior probability is a type of conditional probability in Bayesian statistics.

A posterior probability distribution is a conditional probability distribution obtained by applying the distributional form of Bayes Theorem.

Given a prior belief that a probability distribution function is $p(\theta )$ and that the observations $x$ have a Likelihood $p(x|\theta )$, then the posterior probability is defined as

$p(\theta |x)=\frac{p(x|\theta )}{p(x)}p(\theta )$

where $p(x)$ is the normalizing constant and is calculated as $p(x)=\int p(x|\theta )p(\theta )d\theta$ for continuous $\theta$, or by summing $p(x|\theta )p(\theta )$ over all possible values of $\theta$ for discrete $\theta$.

Posterior vs. Likelihood?

See below

Likelihood

$\mathcal{L}(\theta ;x)=P(x\mid \theta )$

• Measures how likely the observed data $x$ is given parameter $\theta$
• Treats $x$ as fixed and varies $\theta$
• Used in model fitting, e.g., Maximum Likelihood Estimation (MLE)

Posterior

$P(\theta \mid x)=\frac{P(x\mid \theta )\cdot P(\theta )}{P(x)}$

• Probability distribution of parameter $\theta$ after observing data $x$
• Combines likelihood with prior belief $P(\theta )$ using Bayes’ Theorem
• Used for statistical inference and uncertainty estimation

Related

• Prior Probability