Scaling Vision Transformers

Used by V-JEPA and V-JEPA 2.

• ViT-B/L → Original ViT, DeiT, BEiT, DINO, SimMIM, MAE.
• ViT-H → Google’s “Scaling Vision Transformers” (also used in DINOv2).
• ViT-G → JEPA and other foundation models (e.g., OpenAI CLIP-G, Meta I-JEPA).

Citation

(Zhai et al., 2021)

Zhai, X., Kolesnikov, A., Houlsby, N., & Beyer, L. (2021). Scaling Vision Transformers. https://arxiv.org/pdf/2106.04560