Mean Squared Bellman Error Mean Squared Bellman Error: L(ϕ,D)=(s,a,r,s′,d)∼DE(Qϕ(s,a)−(r+γ(1−d)maxa′Qϕ(s′,a′)))2