How does Q-Strata choose bitwidths for each expert in a Mixture-of-Experts model?
A new paper on arXiv describes Q-Strata, a bi-level allocator that assigns different bitwidths to linear layers inside every expert of every Mixture-of-Experts block. For anyone running models such as Mixtral-8x7B-Instruct, Qwen1.5-MoE-A2.7B, or DeepSeek-V2-Lite, the method reduces WikiText2 perplexity below what uniform-bitwidth GPTQ and the state-of-the-art MoE MPQ methods MxMoE and GEMQ achieve in the low-bit regime.
The allocation problem grows with every expert
Mixed-precision quantization assigns a different bitwidth to each linear layer of a large language model. The goal is to minimize quality loss under a fixed budget. In a dense model the number of layers is manageable. In a Mixture-of-Experts model the same layers appear inside every expert of every MoE block. The allocation space grows far larger than in a dense model.
Existing methods handle this growth in two ways. One allocates within each block under a uniform per-block budget. The other allocates across blocks through an additive proxy. Neither directly optimizes a model-level objective over the choices that couple the blocks.
A bi-level allocator with a cached frontier
Q-Strata introduces a bi-level allocator. Its inner stage ranks within-block assignments with a cheap proxy. It caches a Pareto frontier of candidates per block over finely spaced budgets. The outer stage then sets one budget per block instead of a bitwidth for every linear layer.
With the search reduced to one budget per block, the outer stage optimizes the model-level objective directly. This captures the inter-block coupling that additive proxies miss. The assembled quantized model is evaluated during this outer optimization.
Results on three MoE models
The authors test Q-Strata on Mixtral-8x7B-Instruct, Qwen1.5-MoE-A2.7B, and DeepSeek-V2-Lite. They compare against uniform-bitwidth GPTQ and the state-of-the-art MoE MPQ methods MxMoE and GEMQ. Q-Strata consistently achieves lower WikiText2 perplexity in the low-bit regime.
Submitted 31 Aug 2026. Accepted as an EMNLP 2026 Long Paper for the Main Conference. Code is available at the URL provided in the paper.
Gaps the paper leaves open
No latency measurements. No memory footprint after quantization. The cheap proxy construction and its computational cost are not described. The Pareto frontier caching mechanism is not detailed beyond its existence. For deployment decisions, those gaps remain open.
The sources this article was written from
- "arXiv:2608.30564v1
https://arxiv.org/abs/2608.30564v1"