Gaussian Error Linear Units (GELUs)
Used by V-JEPA for how they do attentive probing.
Citation
(Hendrycks & Gimpel, 2016)
Hendrycks, D., & Gimpel, K. (2016). Gaussian Error Linear Units (GELUs). https://arxiv.org/pdf/1606.08415
Used by V-JEPA for how they do attentive probing.
(Hendrycks & Gimpel, 2016)