Muon is Scalable for LLM Training

https://kellerjordan.github.io/posts/muon/#why-is-it-good-to-orthogonalize-the-update

Citation

(Liu et al., 2025)

Liu, J., Su, J., Yao, X., Jiang, Z., Lai, G., Du, Y., Qin, Y., Xu, W., Lu, E., Yan, J., Chen, Y., Zheng, H., Liu, Y., Liu, S., Yin, B., He, W., Zhu, H., Wang, Y., Wang, J., … Yang, Z. (2025). Muon is Scalable for LLM Training. https://arxiv.org/abs/2502.16982