Bayes’ Theorem §

Bayes’ Rule is an application of the LOTP with the Conditional Probability rule. Bayes’ theorem describes the probability of an event based on prior knowledge of conditions that might be related to the event.

Let $B_1,\dots ,B_n$ be a partition of $S$ and $A$ be any event, then $P(B_i|A)=\frac{P(A|B_i)\cdot P(B_i)}{P(A)}=\frac{P(A|B_i)\cdot P(B_i)}{{\sum }_{R=1}^{n}P(A|B_R)\cdot P(B_R)}$

$P(B_i)$ $\to$ Prior Probability $P(B_i|A)$ $\to$ Posterior Probability

How is bayes rule derived? Apply the basic rule of Conditional Probability, and leverage the fact that AND is commutative. $P(B_i\cap A)=P(A|B_i)P(B_i)=P(B_i|A)P(A)$ Therefore, you can simplify $P(B_i|A)=\frac{P(A\cap B_i)}{P(A)}=\frac{P(B_i\cap A)}{p(A)}=\dots$

You have one fair coin and a biased coin that lands on heads with a probability of 3 4 . A coin is chosen at random and tossed three times. If we observe three heads in a row, what is the probability that the fair coin was chosen?

Solution: $B_1=$ The fair coin was chosen $B_2=$ The biased coin was chosen $A=$ 3 heads observed in 3 tosses

We want to find $P(B_1|A)$, so we can use Bayes’ Theorem, and calculate $P(A)=P(A|B_1)P(B_1)+P(A|B_2)P(B_2)$

Bayes rule with Conditioning §

From CS287. $P(x|y,z)=\frac{P(y|x,z)P(x|z)}{P(y|z)}$

Related §

• Conditional Probability
• Bayesian Inference