What happens when semantic IDs stop wasting their first token on what everything shares?

Semantic identifiers turn items into short token sequences. Each token narrows the set of candidates until only the right one remains.

Generative retrieval and recommendation systems use these sequences instead of dense vectors. The tokens are built level by level. At each step a codeword is chosen and the difference — called a residual — moves to the next level.

How the old pipeline builds a code

Residual quantization picks the nearest centroid from a codebook. It subtracts that centroid from the current vector. The remainder becomes the input for the next level. The process repeats until the sequence reaches its fixed length.

The method assumes each level captures a new shared pattern. The first level should catch the broadest commonality. Deeper levels handle finer distinctions.

Where the assumption breaks

The authors identify three failure modes. A corpus-wide shared component can fill the entire first codebook. Hard assignment forces every vector into a single bucket even when it sits between two centroids. Full subtraction leaves a ghost of the chosen centroid direction inside the next residual.

These effects compound. The first token wastes capacity on a pattern that distinguishes nothing. The second token inherits a distorted residual. The third token works with an even noisier signal.

What PRQ-KMeans changes

PRQ-KMeans operates in a post-hoc setting. Reconstruction of the original vector is not required. Three modifications replace the standard pipeline.

First, the global mean is removed before any quantization. The first codebook no longer stores the average of all items.

Second, centroids are updated with Top-k similarity-weighted averages. A vector pulls on its k nearest centroids proportionally to similarity. Hard one-hot assignment disappears.

Third, the residual is computed by projection. Only the component along the selected centroid direction is subtracted. The orthogonal part stays untouched.

What the experiments show

Tests cover one large-scale industrial search dataset and four public recommendation benchmarks. PRQ-KMeans ranks first in overall performance across all evaluated tokenizers.

On the industrial dataset HitRate improves by up to 7.4 percent. MRR improves by up to 11.8 percent. The paper does not report whether the gains hold at every sequence length or only at the maximum length tested.

What the paper does not say

Training time and memory overhead are not compared. The industrial dataset size is not disclosed. No ablation isolates the contribution of each of the three changes. The submission date is 25 August 2026.