Temporal Gradient Inversion for Private Trajectory Reconstruction in Embodied Reinforcement Learning

What it is

The paper names its attack TRACE (Temporal Reconstruction Attack on Consecutive Encodings), an amortized temporal gradient-inversion attack against embodied RL agents that share per-step policy gradients to an aggregating server (robot fleets, shared navigation) while keeping raw observations on-device. Authors are from University of Kentucky and Washington University in St. Louis (arXiv:2609.30258v1, preprint dated 24 Sep 2026). Unlike the existing summary's inference, this is a concrete, evaluated system with theory and numbers.

How it works

TRACE processes a full trajectory of gradients {g1,...,gT} rather than inverting each independently. Three components: a gradient encoder compressing million-dimensional updates to low-dim latents, a causal (masked) transformer that propagates scene context across timesteps, and a decoder with two heads that jointly reconstructs the RGB observation and predicts the discrete action per step. It exploits two signals prior single-frame attacks ignore: cross-time correlation between successive gradients (formalized via a conditional mutual-information bound), and closed-form action recovery from policy-head gradient structure (proven exact when entropy regularization is small).

Two theoretical results matter. Proposition 4.1: at the broadcast checkpoint θ0, A2C and PPO produce the same communicated gradient, which decomposes into policy, value, and entropy terms — so the attack applies to both unmodified. Theorem 4.2: the discrete action label is recoverable from column sums of the policy-head gradient. Training uses teacher-forced warmup, then scheduled sampling plus an auxiliary rollout loss to counter exposure bias.

Preconditions / Affected

• Distributed/collaborative RL with per-step policy-gradient sharing (A2C or PPO actor-critic).
• Honest-but-curious server with white-box knowledge: architecture, current checkpoint θ0, action space, learning rule — but not the data. Attacker trains offline on auxiliary trajectories from the same task family.
• The authors note multi-step client updates (Δθ = θ0 − θ_client) combine gradients along a changing path and may reduce the temporal structure the attack relies on — the per-step assumption is load-bearing.

Results

• 18.8 dB PSNR reconstruction with near-perfect action recovery, at 3–4.5 ms per frame — orders of magnitude faster than optimization attacks, beating learning-based baselines on all metrics.
• Zero-shot transfer to held-out scenes; rapid improvement after adapting on a small subset of target-domain trajectories.
• Generalizes beyond the CNN victim: recurrent, residual, and compact transformer architectures, multi-modal inputs, and larger discrete action spaces (App. F.3).

What to do

The defense finding is the actionable engineering takeaway: standard per-gradient defenses were tested and temporal leakage persists — protecting ordered-gradient streams "may require sequence-aware privacy mechanisms," not just per-step noise. Secure aggregation prevents seeing individual gradients but the authors argue it carries heavy crypto and extra system assumptions that don't fit the ordered-gradient setting. If you run distributed embodied RL, per-step gradient sharing should be treated as a trajectory-disclosure channel (camera views, room layout, movement patterns), and mitigations evaluated against temporal (not single-frame) inversion.

Caveats

Evaluation is on simulated embodied scenes; PSNR of 18.8 dB is a recognizable-but-imperfect reconstruction, not pixel-perfect. The exactness of action recovery is conditioned on small entropy regularization and the stated assumptions (separate actor/critic heads, non-zero advantage, non-degenerate policy). This is a preprint; results are not peer-reviewed.