Cross-Embodiment Gap

This the gap between humans and robots. Once we can effectively figure out how to close this gap, and extract poses from humans, then we can easily retarget it for robots.

• There’s of course the camera perspective

Where does this gap come from?

• Dynamics mismatch (different forces, humans have tactile)
• Perception mismatch (we have Active Perception)
• Different joint lengths

From The authors identify three main “gaps” between human and robot data which their system must bridge:

• Kinematic / embodiment gap (differences in robot vs human arm structure, degrees of freedom)
• Distribution / action gap (differences in how humans move vs how the robot can move)
• Visual / appearance gap (differences in how the robot’s arms appear vs human hands in images)

Some interesting ideas: Flow as the CrossDomain Manipulation Interface.

Human-Robot Embodiment Gap

At least for full-size humanoid, it’s mostly solved? See pages in Locomotion.

• https://huggingface.co/datasets/lvhaidong/LAFAN1_Retargeting_Dataset
• https://xskill.cs.columbia.edu/

Robot-Robot Embodiment Gap (X-Embodiment)

If you look at the models released

• https://sites.google.com/view/xembodimentworkshop

Should really look through data