Sachin

Met at the Comma hackathon.

Start with Camera Calibration using Zhang’s methods

• Know the basics about Kalman Filter, Cyrill Stachnis

Pre-req to ORB-SLAM: Fundamental Matrix and Homography Matrix Epipolar Geometry

Types of SLAM

1. Sparse/ (Feature based)
2. Sparse (but intensity based), called semi-direct method (SVO)
   1. Source: https://rpg.ifi.uzh.ch/svo_pro.html
3. Dense (ElasticFusion like https://www.roboticsproceedings.org/rss11/p01.pdf), COLMAP fits under this as well
   • LSD-SLAM

We also say like indirect (feature-based) and direct SLAM (intensity based)

Tip from sachin: Focus on Sparse, because dense usually builds on top of sparse.

For robotics, 1-2cm tolerance, but in VR they need SUPER SUPER tight tolerance

https://github.com/MIT-SPARK/Kimera-VIO

4. ORB-SLAM: track features

Projective 3-Point Algorithm (for marker tracking, infer camera position) https://www.youtube.com/watch?v=N1aCvzFll6Q&list=PLgnQpQtFTOGRYjqjdZxTEQPZuFHQa7O7Y&index=38&pp=iAQB&ab_channel=CyrillStachniss

• http://introlab.github.io/rtabmap/
• https://github.com/uzh-rpg/rpg_svo_pro_open

So it seems that there are 2 playlists that are super helpful:

• Photogrammetry 1 and 2: https://www.youtube.com/playlist?list=PLgnQpQtFTOGRYjqjdZxTEQPZuFHQa7O7Y
• Mobile sensing 2: https://www.youtube.com/playlist?list=PLgnQpQtFTOGQh_J16IMwDlji18SWQ2PZ6

Actually Mobile sensing 1 has some pretty good videos too: https://www.youtube.com/playlist?list=PLgnQpQtFTOGQEn33QDVGJpiZLi-SlL7vA

• Includes MPC and Kalman Filter

Start with this: (understand backend optimization)

• https://www.youtube.com/watch?v=r2cyMQ5NB1o&pp=ygUUbGVhc3Qgc3F1YXJlcyBjeXJpbGw%3D
• https://www.youtube.com/watch?v=z60RbiY18I8&ab_channel=CyrillStachniss

For Cyrill (follow the order):

1. Camera Calibration https://www.youtube.com/watch?v=uHApDqH-8UE&list=PLgnQpQtFTOGRYjqjdZxTEQPZuFHQa7O7Y&index=32&ab_channel=CyrillStachniss
2. (optional, interested only) https://www.youtube.com/watch?v=QwU7iSJK8Rk&list=PLgnQpQtFTOGRYjqjdZxTEQPZuFHQa7O7Y&index=10&pp=iAQB&ab_channel=CyrillStachniss
3. (this is different from Feature Matching, it’s just correlation between two images, not really needed) https://www.youtube.com/watch?v=5YAA7vS6kVU&list=PLgnQpQtFTOGRYjqjdZxTEQPZuFHQa7O7Y&index=12&ab_channel=CyrillStachniss https://docs.opencv.org/4.x/d4/dc6/tutorial_py_template_matching.html
4. Feature Points (part 1 and 2)
5. Homogeneous (quickly glance, see if you understand) https://www.youtube.com/watch?v=MQdm0Z_gNcw&list=PLgnQpQtFTOGRYjqjdZxTEQPZuFHQa7O7Y&index=30&ab_channel=CyrillStachniss
6. DLT (super important): https://www.youtube.com/watch?v=3NcQbZu6xt8&list=PLgnQpQtFTOGRYjqjdZxTEQPZuFHQa7O7Y&index=35&ab_channel=CyrillStachniss
7. Zhang’s method, https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html

https://www.youtube.com/watch?v=3NcQbZu6xt8&list=PLgnQpQtFTOGRYjqjdZxTEQPZuFHQa7O7Y&index=35&ab_channel=CyrillStachniss