How Does a Gradient Stream Reveal What a Robot Saw?

How the Attack Reads Time

The attack exploits two structural signals that single-frame methods ignore. First, cross-time correlation between successive embodied gradients creates a conditional mutual-information bound that ties each step to the next. Second, the policy-head gradient structure allows closed-form action recovery when standard entropy regularization is sufficiently small. TRACE autoregressively reconstructs the sequence frame by frame. Prior work treated each gradient as an independent snapshot. The authors formalize why that assumption fails for embodied agents. Sensor streams from a moving body carry strong temporal dependence. That dependence survives the gradient computation and reaches the server.

Why Speed Changes the Threat Model

The method is amortized. A learned inverter maps gradients to observations without per-sample optimization. This design choice explains the speed advantage reported later. The inverter is trained offline on public data. During the attack it runs in a single forward pass per step. No iterative optimization is needed at inference time. On held-out embodied scenes TRACE reaches 18.8 dB PSNR with near-perfect action recovery. Each reconstructed frame takes 3-4.5 milliseconds. The method dominates the learning-based baseline across all reconstruction metrics. It also exceeds optimization attacks. Speed is the striking difference. TRACE runs orders of magnitude faster than optimization approaches while achieving better fidelity.

Where Evaluation Extends

Evaluation covers recurrent architectures. Residual architectures are also covered. Compact transformer victim architectures round out the test suite. Multi-modal inputs and larger discrete action spaces are also covered. The paper was submitted on 24 Sep 2026. It was accepted at NeurIPS 2026. The arXiv identifier is 2609.30258v1.

What Defenses Must Now Address

Defense experiments suggest that protecting temporal gradient streams may require sequence-aware privacy mechanisms. Standard differential privacy applied per step does not address the cross-time correlation. The authors do not specify which sequence-aware mechanisms would suffice. Injury data is not reported. For anyone deploying embodied agents that share gradients, the implication is clear. The raw sensor stream never leaves the device, yet the gradient stream carries enough temporal structure to reconstruct it. Sequence-aware defenses are now a research priority.