Sim2Real
The “sim2real” problem refers to the challenge of transferring knowledge and skills learned in simulation environments to the real world.
- this problem exists arises because simulations can never fully capture the complexity and variability of the real world
Idea: Learn to race in a video game, and translate those skills in real life.
Ways to bridge the Sim2Real gap (source):
- System identification: build a mathematical model for a physical system; (this is the simulator in the context of RL)
- Domain Adaptation (adversarially generated environments)
- Domain Randomization
Both DA and DR are unsupervised. Compared to DA which requires a decent amount of real data samples to capture the distribution, DR may need only a little or no real data. So Domain Randomization might be better.
repo NVIDIA “Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation” at NeurIPS 2021:
Synthetic Data Generation at WATonomous
Ideas:
This I don’t know how I got this. I was looking through BEVFusion, and then found LSS which the paper mentioned, which was written by Jonah Philion. And then I stalked his advisor who is prof. Sanja Fidler.
But the professor Sanja Fidler, I was browsing through her research, and found this Meta-Sim: Learning to Generate Synthetic Datasets
- These set of slides seem good https://www.cs.utexas.edu/~yukez/cs391r_fall2021/slides/pre_10-05_Changhan.pdf