Gaussian Error Linear Units (GELUs)

Used by V-JEPA for how they do attentive probing.

Citation

(Hendrycks & Gimpel, 2016)

Hendrycks, D., & Gimpel, K. (2016). Gaussian Error Linear Units (GELUs). https://arxiv.org/pdf/1606.08415