Prediction

https://medium.com/cruise/cruise-continuous-learning-machine-30d60f4c691b

• I think you should take inspiration from how prediction is done, https://tisl.cs.toronto.edu/publication/201902-its-inferring_pedestrian_motions/its19-inferring_pedestrian_motions.pdf

Papers

• Deep Learning-based Vehicle Behaviour Prediction for Autonomous Driving Applications: a Review

“Prediction can be autolabelled” $\to$ https://medium.com/cruise/cruise-continuous-learning-machine-30d60f4c691b

Prediction is nice because you get your labels from Perception. First introduced by Eric Lin from Cruise over our coffee chat, but it seems I also heard this elsewhere.

You should work on this prediction challenge from Nuscenes: https://www.nuscenes.org/prediction?externalData=all&mapData=all&modalities=Any

Trajectron++

https://arxiv.org/pdf/2001.03093.pdf

Use a VAE, and use the latent space. Each node represents a particular behavior (a modality).