Laplace Distribution
First saw reading this article https://semianalysis.com/2024/01/11/neural-network-quantization-and-number/
They’re essentially very sharp Gaussian Distributions.
“In practice, neural networks are typically normal or laplace distributed”.