Distribution

Multinomial Distribution

title: Multinomial Distribution (Definition)
We say the vector $(X_1, X_2, . . . , X_k ) ∼ Multi(n, p_1, p_2, \dots, p_k)$ follows a Multinomial Distribution if:  
1. There are $n$ trials, where each trial has $k$ possible outcomes with probabilities $p_i$, with $i = 1, 2, \cdots, k$ and $\sum_{i=1}^k p_k = 1$
2. The trials are independent.
3. $X_i$ is the number of successes of type $i$ in $n$ total trials, so $\sum_{i=1}^k X_i = n$

The joint pmf is given by $f(x_1,x_2,\dots ,x_n)=\frac{n!}{x_1!x_2!\dots x_n!}{p}_1^{x_1}{p}_2^{x_2}\dots p_n^{x_n}$

Marginals $f_{x_1}(x_1)=\left(\genfrac{}{}{0ex}{}{n}{x_1}\right)p_1^{x_1}(p_2+\cdots +p_n{)}^{x_2+\cdots +x_k}$

Conditionals I a little iffy about what this means, they gave this example, but I don’t think it’se worthy to put it in my notes.

PyTorch

Serendipity learned this from STAT206, and now I am applying it from Andrej Karpathy.

torch.multinomial(p, num_samples=20, replacement=True, generator=g)

We use the probability distribution to generate a tensor of indices.