Stochastic/Markov Game

Stochastic Games generalize MDPs to multiple interacting decision-makers.

Mentioned in F1TENTH Research Proposal.


A Markov Game is a tuple

  • where is a finite set of states for player
  • where is a finite set of actions for player
  • is a state transition probability matrix
  • is a reward function, where
  • is a discount factor,

Value Function

Nash Equilibrium A Nash Equilibrium of the Markov game is a joint policy , such that for any and

  • where represents the indices of all agents in N except agent .