ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters

I remember seeing this paper in 2023 through two minutes papers. Now, I actually have more ML background to fully understand it.

But the paper is actually quite technical.

Most of the pioneering work was done by DeepMimic. But that method mostly works for single tasks (backflip, walking, etc.)

This paper is an attempt at teaching reusable general-purpose motor skills, rather than training policies from scratch for each specific task. That’s why they actually split it into high-level and low-level policy.

• The architectures are really simple

Interestingly, they use the JS Divergence, but since computing that is expensive, they use a variational approximation

This is their learning objective: ${\max }_{\pi }-D_{JS}(d^{\pi }(s,s^{\prime })\parallel d^M(s,s^{\prime }))+\beta I(s,s^{\prime };z|\pi )$

• First term is using the JS Divergence to learn a policy that learns the reproduce the entire distribution of behaviors from the motion dataset $M$
• Second term is a “skill discovery” objective

Project page:

• nv-tlabs.github.io/ASE/

Follow up paper:

• CALM Conditional Adversarial Latent Models for Directable Virtual Characters

Related

• VideoMimic