Belief State

I think this is discussed in the case of POMDP, where we maintain some sort of belief state, because we don’t know what exact state we are in.

Note

A belief state $b(h)$ is a probability distribution over states, conditioned on the history $h$ $b(h)=(\mathbb{P}[S_t=s^1|H_t=h],\dots ,\mathbb{P}[S_t=s^n|H_t=h])$