Redirecting to original paper in 30 seconds...
Click below to go immediately or wait for automatic redirect
📄 Abstract
Abstract: We study neural network compressibility by using singular learning theory to
extend the minimum description length (MDL) principle to singular models like
neural networks. Through extensive experiments on the Pythia suite with
quantization, factorization, and other compression techniques, we find that
complexity estimates based on the local learning coefficient (LLC) are closely,
and in some cases, linearly correlated with compressibility. Our results
provide a path toward rigorously evaluating the limits of model compression.
Key Contributions
JSON parse error: Unexpected token s in JSON at position 29506