data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language

Similar to I-JEPA.

https://ai.meta.com/blog/ai-self-supervised-learning-data2vec/

Citation

(Baevski et al., 2022)

Baevski, A., Hsu, W.-N., Xu, Q., Babu, A., Gu, J., & Auli, M. (2022). data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language. https://arxiv.org/pdf/2202.03555