Why does model merging fail when task vectors point the wrong way?

Model merging by task arithmetic works until it doesn't. The field has spent years blaming magnitudes: layerwise representation bias, deviations from cross-task linearity, parameter overlap.

A new paper tracks the exact layerwise cross-term of merged LLMs through a factorial ledger and intervenes on it directly. Magnitude proves insufficient — and inconsistent across model families — as a diagnostic axis.

The flux is not in the numbers

An exact decomposition of the layerwise flux shows it is dominated by amplifying transport of the existing cross-term. That transport accounts for ~65-70% in both families. Gain exceeds 1 per late block. Erasing the term is undone by propagation. The system rebuilds it to 99% of its norm at cosine 0.99 unless the erasure sits near the output.

Direction carries the weight

A basin test with six starting displacements establishes the carried direction as an attractor of the forward pass. That direction is causally load-bearing. Erasure along it removes expressed interference dose-dependently and saturates at exact erasure. Norm-matched wrong-direction controls fail or backfire.

Wrappers drown the signal

Instruction wrappers gate the effect. The same erasure finds 13x less relative interference to remove under a wrapper that internally amplifies the cross-term. The wrapper drowns the interaction in a template-pinned main effect rather than shrinking it. This structure replicates across further instruction templates but not under a length-matched control.

Magnitude is a coarse correlate

The striking +-15% "universality" of naive bfloat16 generation turns out to be quantization roughness. Task pairs whose local cross-term generation differs by at most 1.9x differ by 14x-337x in causally removable interference.

All 46 predictions were preregistered and frozen before their data. Falsifications — including of the authors' own headline expectations and of behavioral recovery under a validated continuous endpoint — are reported as such.

The paper does not say whether the attractor direction can be predicted before merging. It does not say whether the 13x wrapper effect holds for every prompt style. It does not say how to choose the erasure layer in practice.