How does QuantForge find better four-bit quantization recipes without human trial and error?

Why four-bit quantization stays hard

Four-bit post-training quantization can shrink the memory footprint of large language models. The MXFP4 W4A4 format demands that several design choices work together at once. Coordinate transforms change block-encoding errors. Those errors then reshape the residuals that flow through the network. Because the useful algorithmic decomposition is not fully known before search, engineers cannot simply pick a recipe off a shelf. LLM-driven program evolution offers a way to explore the space. Performance scores alone do not explain which design should change next. A higher number may come from a lucky accident rather than a principled improvement. Without an explanation, the next mutation is a guess.

The core problem

Explanations matter more than scores.

How QuantForge runs controlled experiments on code

QuantForge records competing explanations for why a program works. It then selects controls that distinguish those explanations. The system checks that successor code actually implements the resulting conclusions. This residual compilation guides program revisions while retaining useful programs even when their original explanations are rejected. Remeasuring the revised program reveals the next error to address. The loop continues until the budget of evaluator calls is spent.

The experimental proof

In matched-budget comparisons each run received 240 evaluator calls. QuantForge reached a held-out transfer target in six of eight runs. Textual memory and reflection memory each succeeded three times. Score-only evolution succeeded once. QuantForge also evaluated fewer new programs than the baselines.

HiRes works in three stages

The process discovers HiRes, a fixed MXFP4 quantizer. HiRes shapes coordinates in its first stage. The second stage refines legal code assignments. The third stage recovers errors along attention and MLP paths. Each stage acts on residuals measured after the preceding stage has executed.

Results and limits

Across seven tasks HiRes achieves the lowest seven-model Robust Fit at 0.09300. It also achieves the lowest quantized Fit-7 at 32B. The paper does not report latency or energy measurements for the discovered quantizer. It also does not describe how the method behaves on models larger than 32B.

What this changes

Turning controlled evidence into subsequent program changes improves the discovery of transferable PTQ algorithms. The experiments cover seven tasks and a single model size. Whether the same loop scales to larger models or different quantization formats remains an open question. The source code and artifacts are linked from the arXiv entry for further inspection.