How does ProvenanceGuard catch when an MCP agent cites the wrong source?

An MCP agent can call a search tool, read a patient record, query a database, and then weave everything into one answer. Most factuality checkers only ask whether the pooled evidence supports the claim.

ProvenanceGuard adds a different question: does the specific source the answer cites actually back that claim?

The failure mode has a name

Cross-source conflation happens when a claim is true in one tool output but the answer attributes it to another. A customer-support agent might say the account record promises a 30-day refund window. The window exists in a policy document. Pool the two and the claim looks supported. Keep them separate and the attribution is wrong. In a clinical agent a medication detail from a patient-history tool becomes misleading the moment the answer presents it as a finding from the medical literature.

Source-blind verifiers pass the claim because the fact sits somewhere in the soup. ProvenanceGuard keeps the connection between claim and source visible for inspection.

What the layer does after generation

The verifier sits on top of a black-box MCP agent. It reads the captured MCP trace — tool outputs with their source IDs — without retraining the agent. Then it runs five steps in sequence. It breaks the answer into specific claims. It finds the source most relevant to each claim. It checks whether that source actually supports the claim. It compares the supporting source with the one the answer names or implies. Finally it emits a per-claim source verdict and a global allow-or-block decision.

Source identity travels through decomposition, routing, support scoring, attribution checking, and repair. It never collapses into one anonymous context.

The models used in the reported experiments

For the paper's tests the team ran everything locally so traces stayed in a controlled offline setup. MiniLM helped find the relevant source. A DeBERTa NLI verifier model checked whether that source supports the claim. A local language model helped break answers into claims. The verifier also checks literal values closely: a number, date, or identifier absent from the source cannot pass merely because the sentence sounds plausible. A calibrated decision step combines these signals.

Those named models are the setup they evaluated, not a requirement of ProvenanceGuard. The same claim, source, and decision steps can be adapted to hosted models where a team prefers cloud services; a new setup would need its own testing and calibration.

Results from 281 medical traces

The medical agent used patient records, research articles, and other tools. Medicine is a useful test because a fact from a patient's record and a fact from general research cannot be treated as the same source. Human experts checked 361 claims from 40 answers held out from the development data.

Experts said 139 claims should not pass. ProvenanceGuard caught 138 of them. It let one through. It also held 67 claims the experts considered supported, sending them for review or repair. This reflects the cautious setting tested: it favors a second look at some supported claims over letting unsupported ones through.

For claims with an identifiable source the system picked the right source about 86 percent of the time.

Comparison with source-blind baselines

Four other support checkers ran on the same claims. ProvenanceGuard scored highest on the paper's measure of how well a system catches claims that should be blocked while avoiding unnecessary blocks.

The other checkers did not tell which tool output supported each claim. ProvenanceGuard records that connection so a reviewer can see the source checked for each claim and the decision it produced.

When sources look similar

In a separate harder test with several similar sources ProvenanceGuard scored 0.846 F1 for deciding which claims to block but identified the exact source correctly in 50.3 percent of claims. Telling similar sources apart remains an important area for improvement.

The team also ran a controlled test focused on wrong attribution. They changed the named source in 50 cases while leaving the supporting evidence intact. ProvenanceGuard caught all 50 swaps. This shows it can detect a clear source error while the harder test shows the challenge of choosing among many plausible sources.

Repairing blocked answers

Blocking is only useful if there is something to do with a blocked answer. Wired to a RARR-style repair loop the full-trace run resolved all 173 blocked answers though 144 of them ended in fallback text rather than a substantive rewrite. The system chose to avoid an unverifiable answer rather than manufacture one. On reconstructed multi-source test traces a fresh repair run resolved all 59 initially blocked answers with only two terminal fallbacks.

As an offline gate the overhead is modest, roughly half a second per answer on the reported local configuration, with the NLI and routing calls themselves in the tens of milliseconds.

Where else the approach appears

NVIDIA NVFlow merged an optional grounding-verification stage for its finance agent. It checks completed answers against the SEC excerpts the agent retrieved and saves separate decisions without changing the original rollout or training data. The NVFlow contribution uses ProvenanceGuard's source-aware verification approach; the repair loop discussed above belongs to the broader research system.

This work was also presented as a poster at the Agentic AI Summit 2026 at UC Berkeley.

What the paper does not cover

The announcement does not say how many of the 281 traces came from each tool type. It does not report latency for a hosted-model configuration. It does not describe how the calibrated decision threshold was chosen beyond calling it conservative. It does not compare the repair quality of substantive rewrites versus fallbacks beyond the fallback counts.