Robot Foundation Models

π0: A Vision-Language-Action Flow Model for General Robot Control

Links:

• https://www.physicalintelligence.company/blog/pi0

The images and proprioceptive state are encoded via corresponding encoders and then projected via a linear projection layer into the same embedding space as the language tokens.

Model Architecture

• siglip
• PaliGemma

” averaging over 10 trials per task”

• This is how many trials they do to get success rate

Why flow-matching?

• To ensure /constrain smooth robot outputs as opposed to random jumps in values

VLM Logic

LLMs predict sequences of tokens. What does a VLM predict?

Flow Matching Math

At training time, the Flow Matching loss to train the policy is given by $L_{\tau }(\theta )={\mathbb{E}}_{p(A_t\mid o_t),q(A_t^{\tau }\mid A_t)}{\Vert v_{\theta }(A_t^{\tau },o_t)-u(A_t^{\tau }\mid A_t)\Vert }^2$

Where

• $u(A_t^{\tau }\mid A_t)=ϵ-A_t$ (Note: this should be $A_t-ϵ$?)
• $A_t^{\tau }=\tau A_t+(1-\tau )ϵ$

Potential source of confusion

Notice that $v_{\theta }(A_t^{\tau },o_t)$ is always learning to predict $ϵ-A_t$, even though it is conditioned on $A_t^{\tau }$. You might think really it should be learning $\tau (ϵ-A_t)$, but you’d be mistaken.

• This multiplication by $\tau$ will be done at inference time to control “step size”

What's the point of \tau?

Without $\tau$, where you just start from $ϵ$ and directly predict $A_t$, that’s essentially a denoising autoencoder view.

$\tau$ allows you to better capture multi-modal behavior, else you end up with mode-collapse, think about this scenario:

Flow matching trains on all intermediate points: $A_t^{\tau }=\tau A_t+(1-\tau )ϵ,\tau \in [0,1]$ Taking the derivative with regards to $\tau$, we see that: $\frac{d}{d\tau }{A}_t^{\tau }=\frac{d}{d\tau }(\tau A_t+(1-\tau )ϵ)=A_t-ϵ$

• The derivative of a straight line is a constant slope

We learn to predict this gradient (a constant) so that at inference time, we learn this mapping for any arbitrary $A_t^{\tau }$ $\frac{d}{d\tau }{A}_t^{\tau }=v_{\theta }(A_t^{\tau },o_t)=A_t-ϵ$

At inference time, they start with random noise $A_0^t\sim \mathcal{N}(0,I)$ and integrate the learned vector field from $\tau =0$ to $\tau =1$. They and use the forward euler integration rule: $A_t^{\tau +\delta }=A_t^{\tau }+\delta v_{\theta }(A_t^{\tau },o_t)$

• where $\delta$ is the integration size ($\delta =0.1$ in paper)

Why is 10 steps better than 1 step?

Because at the end of the day, we are learning a vector (a line essentially).

• ChatGPT answer: If you take 10 smaller steps, each step only needs to be locally correct. Integration keeps pulling you back onto the line. So error doesn’t explode; it averages out