A Stochastic–Geometric Theory of Scaling Laws in Grokking
Published in Preprint — under review, 2026
Grokking occurs when a model memorizes training data long before it begins to generalize. This work characterizes a shell–core geometry of the solution space induced by Adam with weight-shrinkage regularization. Using stopping-time theory to study the transition from memorization to generalization, we derive scaling laws for learning rate, batch size, and regularization strength, and validate them experimentally.
Read the paper on arXiv.
Recommended citation: Luo, R., Gagné, C., Ngnawé, J., Ullah, I., & Morrissey, K. (2026). A Stochastic–Geometric Theory of Scaling Laws in Grokking. arXiv preprint arXiv:2606.30388.
Download Paper