How does splitting logic from persona protect small models on edge devices from context pollution?

Researchers from AO, Inc. and OrbLabs AG published a preprint that separates reasoning from style in small language models running locally. The architecture keeps the logic prompt tiny and stable while the persona adapter swaps in milliseconds.

One context window, two jobs

A small model on a laptop or phone must sound like a chosen character and produce correct structured output at the same time. Both demands share one context window that grows with every turn, every system instruction, and every few-shot example the developer stuffs in.

The interference problem

When the history gets long and persona-heavy, the logical part of the answer starts to break. The paper calls this persona-logic interference. Logic fails. In a standard single-pass prompt the model sees the full conversation plus the persona description plus the task specification all at once. On Llama-3.1-8B-Instruct the prompt balloons from 242 tokens at the cleanest level to 1,203 tokens at the dirtiest. On Gemma-3-4B-it the same growth happens. The composite logic score drops from 0.669 to 0.150 on Llama and from 0.487 to 0.150 on Gemma. Most failures are not wrong reasoning but missing or malformed structured output: 80 to 95 percent of runs on Llama and 100 percent on Gemma at the two highest pollution levels.

Decoupling architecture

AO-DA splits the work into two inference paths on the same INT4 base model. The logic path receives only the current core turn and emits a verifiable Micro-State. Its prompt stays fixed at 180 tokens for Llama and 167 tokens for Gemma regardless of pollution level. The outputs are byte-identical across all four levels in 40 out of 40 runs. The persona path takes that Micro-State plus the full history and renders the final reply in character. A dedicated LoRA adapter handles each path; they hot-swap in 1.7 milliseconds.

What the numbers show

On the 8B model the decoupled logic path fails in 0 to 20 percent of runs versus 80 to 95 percent for the single pass. Paired improvement Δ ranges from +0.30 to +0.50, Cliff's δ from 0.50 to 0.85, Holm-adjusted p ≤ 0.03. On the 4B model both approaches collapse. The paper does not say why the smaller model cannot benefit from the same separation.

Cost and trade-off

The first turn of a topic takes one extra decode: 28.2 seconds versus 18.2 seconds on Llama on an Apple M2 laptop. After that the logic path is reused and only the persona adapter swaps. The released artefacts include code, scoring rubric, pollution fixtures, adapters, and full run logs. How many devices currently run such agents in production is not mentioned in the preprint.