What Changes When Quantized Models Get a Complexity Score Based on Behavior?
A research paper posted on arXiv introduces Behavioral Capacity Certificates, a way to measure the complexity of quantized language models by what they actually compute rather than by the codes stored in their weights. The method treats two deployments as equivalent when they produce the same bounded loss, even if their weight codes differ.
Why Weight Codes Are Not Enough
Quantization changes the precision of activations and key-value caches. Those changes alter what a model computes without touching the stored weights. Traditional complexity bounds look only at the weight codes. They assign the same score to deployments that behave differently. They also charge separately for weight codes that behave identically. The mismatch matters when engineers choose bit-widths per layer or decide how to split precision between keys and values.
The Certificate Counts Implementations Differently
Behavioral Capacity Certificates charge for behavior using the aggregate prior mass of complete implementations — weights, scales, activation rules, and cache rules. All implementations that induce the same bounded loss share one certificate. When quantization merges distinct implementations, the shared mass lowers the complexity penalty. A break-even law then decides whether the saving survives the cost of validating it.
Three Steps From Screen to Bound
The authors describe a deployment workflow with three steps. First, a forward-only screen shortlists per-layer bit-widths by how often candidate perturbations preserve the reference predictions. Quality matches Hessian-guided selection at lower preprocessing cost. Second, margin-certified cells identify weights that can be pruned or sign-flipped without changing the deployed behavior. Every permitted combination preserves all declared predictions. This holds. On OLMoE-1B-7B and SmolLM2-1.7B, independent probes bound the probability that any permitted combination changes a prediction on new text. Third, BCC bounds the population loss of the deployed model. The bound is nonvacuous for complete decoders and tighter than the compressed-code route.
Keys Versus Values at Equal Cache Memory
At equal cache memory, giving keys higher precision than values yields lower negative log-likelihood and higher prediction agreement. The result holds on GPT-2, Qwen2.5, and SmolLM2. The GPT-2 audit also shows a tighter complexity bound.
Behavior matters.
The sources this article was written from
- "arXiv:2609.37887v1
https://arxiv.org/abs/2609.37887v1"