How does a new method assign bits to neural network layers 2,570 times faster?

A new arXiv paper describes a way to compress neural networks by choosing precision for each layer separately. The method skips the slow search that older approaches need and keeps working even when the calibration data are noisy.

Why mixed precision remains hard

Mixed-precision post-training quantization assigns a bit width to every layer under a single memory budget. The calibration set is small. The search space grows combinatorially. Earlier methods rely on external solvers or iterative heuristics that take hours. They also break down when the calibration samples contain corruptions. The paper frames both problems as consequences of the same gap: no clean separation between the error a layer introduces and the error it merely passes forward.

A separable score that needs no solver

The authors derive a layerwise probabilistic analysis of the quantization error. The derivation splits the total perturbation into a local term and a propagated term. The local term depends only on the layer's own weights and the calibration distribution. Because it is separable, the allocation reduces to a simple greedy ranking. No integer programming. No reinforcement learning. No differentiable relaxation. The algorithm sorts layers by the score and fills the budget from the top. That is the entire allocation loop.

It works.

Speed gains measured on DRUNet

Experiments use DRUNet on denoising tasks with an average budget of 4 bits per weight. Under clean calibration the method matches or improves the state-of-the-art mixed-precision baselines. When the calibration set is corrupted the gap widens. Peak signal-to-noise ratio improves by up to 7.5 dB under the tested corruptions. Bit-allocation speed-ups range from 28x to 2,570x over the studied baselines. The fastest run finishes in seconds.

Diffusion models benefit without redesign

The framework applies directly to quantized diffusion models. No architectural changes. No retraining. The same separable score drives the allocation. Results show an improvement over the prior state-of-the-art on that class of models as well. The paper does not report wall-clock numbers for diffusion experiments.

What changes for a model owner

You can quantize a large network to mixed precision on a laptop in the time it takes to brew coffee. The calibration set can be small and imperfect. The allocation code has no external dependencies. The method does not guarantee a specific accuracy target for every architecture. The authors note that the probabilistic bound assumes a calibration distribution close to the test distribution. How far that assumption stretches in practice is not quantified in the document.