41 results for “Reinforcement learning”

Paper studying whether self-evolution weakens personal agents' robustness to indirect prompt injection (IPI). A two-phase method has the user ask the agent to follow tasks in external content, then tests with a third-party injected task the user did not authorize. Against the non-evolving base agent, attack success rose 14.0 to 48.8 points under skill evolution and 3.0 to 39.3 points under reinforcement learning. Code is released.

arxiv.orgpaperAdded 31 Oct 2026, published 29 Oct 2026🛡️ Security

VideoLVR is a method that lets video multimodal LLMs build latent visual evidence from local regions at relevant moments and reason over it interleaved with text tokens. A visual token prediction objective, defined over a vocabulary derived from the input video, supervises the latents. It extends to reinforcement learning with sampled visual tokens as actions and final-answer rewards, and improves fine-grained video benchmarks without much context growth.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

EnvSched is an online scheduler for agentic reinforcement learning that picks which environment-task pairs to roll out next. It builds a dual-relation graph over environments and tasks, propagates past rollout statistics to predict within-task reward variance, and adds environment quotas and a similarity penalty. On Qwen3-8B it scores 48.81 average on BFCL-v3, tau-Bench and ACEBench-Agent after 40 steps, above random selection at 120 steps.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

NEXUS is a hierarchical reinforcement learning framework that pairs neural skills with symbolic meta-policies, so transparent reasoning sits at the level of high-level skills rather than atomic actions. It uses object-centric observations and LLM interaction to design reward functions and meta-policies. Agents learn disentangled skills, align better with goals in the game manual, resist reward hacking more, and cope better with game simplifications than deep RL. The paper also compares neural, symbolic and neuro-symbolic meta-policy architectures.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

FORK is a reward for rubric-based reinforcement learning that jointly optimizes recall and a new precision objective. Precision requires each unit of a response to be necessary (rubric-driven necessity credits) and factually correct (a grader). Rubric-coverage rewards otherwise produce verbose, less accurate responses. FORK keeps similar coverage with shorter outputs, raising win rate against the base policy from 10.6% to 68.0% on RubricHub-Medical and from 22.1% to 74.1% on HealthBench.

arxiv.orgpaperAdded 27 Oct 2026, published 24 Oct 2026🧠 AI

ProfileSim is a profile-driven user simulator for financial marketing outbound calls. It models user profiles (demographics, Big Five traits) and business profiles (intents reverse-engineered from real call logs). The base model is fine-tuned on real voice dialogues, then refined with GRPO using dialogue-level and utterance-level rewards. Against other user generators it scores highest on colloquial naturalness and profile adherence, and lowest agent score on long-tail intents, with shorter utterances at comparable business compliance.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

MART-GRPO is a variant of group-relative policy optimization for vision-language-action (VLA) reinforcement learning. It uses a LoRA-fine-tuned VLM to find the last visible error without recovery in a failed rollout video, keeps failure feedback up to that point, and drops it afterward. Martingale stopping conditions show this preserves the expected GRPO gradient and can reduce covariance. With OpenVLA-OFT on RoboTwin 2.0, it raises success rates over full-trajectory GRPO across multiple tasks.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

This paper trains small open language models with reinforcement learning to select tools for multi-step instructions over corpora of more than ten thousand tools. A Qwen3-8B model post-trained with the recipe outperforms Llama-4-Maverick (402B) on ToolBench. The trained model transfers to different retrievers and generalizes to new tool corpora in other domains, without relying on proprietary LLMs like Claude or ChatGPT.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

InterWebCoder is a framework for interactive webpage code generation by multimodal LLMs. It synthesizes self-contained webpages with executable test scripts from recorded interaction trajectories, repairs failures using runtime evidence, then trains with test-ID-weighted SFT and GRPO using test pass rates and visual similarity as rewards. The pipeline yields 1,736 verified samples from 2,058 tasks. A 9B model scores 48.54 on IWR-Bench and 48.42 on Vision2Web, and reward latency drops over 97%.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

AgenticRoboRL is a framework for post-training robot planners across different harnesses, combining supervised fine-tuning with multi-turn reinforcement learning. Harness-specific adapters link recorded planner interactions to task outcomes, so one RL pipeline covers both VLA skill invocation and control-code generation. On LIBERO-PRO, a 9B planner rose from 18.1% to 71.0% without external memory, near Codex (GPT-5.5) with memory at 72.1%. Code and checkpoints are promised.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

E-Layout is an ML-driven framework for post-placement logic restructuring in IC physical design. It builds equivalent logic alternatives with e-graph equality saturation, links logic choice to physical realization through a graph-based RAG model, and refines the choices with reinforcement learning on design-level timing feedback. On 9 designs it gives about 6% TNS and 2% WNS improvement and 1% less area than a commercial physical design flow.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

Proposes a steering agent that supervises LLM agents and multi-agent systems during execution and corrects goal drift, where long-horizon tool use and inter-agent collaboration slowly diverge from the overall objective. A low-cost fast path decides when to intervene, and a high-cost slow path generates how to intervene. Both paths are trained jointly with reinforcement learning, combining a pairwise ranking objective with GRPO. On information-seeking search tasks it improves performance and efficiency for single-agent and multi-agent systems, generalizes out of domain, transfers to unseen executors, and allows intervention frequency to be adjusted without retraining.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

Introduces Librarian Agents, where a builder agent turns raw unstructured documents into a hierarchical filesystem of synthetic files and reference links, forming a workspace for a downstream answering agent. Workspace construction is cast as a Markov decision process and the builder policy is trained with reinforcement learning to maximize downstream answering accuracy. It beats the strongest retrieval and precomputation baselines by 33 points on BrowseComp-Plus, amortizes construction cost over repeated queries, and generalizes zero-shot to BRIGHT with a 31-point average gain.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

X-RIC (eXplainable RIC) is a framework that adds agentic reasoning to the O-RAN Near-RT RAN Intelligent Controller. It combines reactive reinforcement learning with automated behavioral cloning from domain heuristics, a transformer-based Telecom Language Model for contextual abstraction and interpretability, and closed-loop co-adaptation for continuous policy refinement. The paper also introduces BARX, a benchmark for agentic RAN control that measures latency, compliance and determinism. On SLA-aware network slicing, in simulation and on an O-RAN-compliant testbed, it reports 89% lower operational cost with SLA compliance kept, 75% faster convergence, 50% fewer training steps and 46% lower variance.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

Proposes a task where a model judges and diagnoses a candidate program from only the problem statement and source code, backing its verdict with generated counterexamples that can be checked by execution. The authors build USACO-JUDGE from USACO 2026 contests (human and LLM-written accepted and unaccepted programs) and train ARBITER, a 35B MoE agentic verifier, with reinforcement learning in a generic execution sandbox. Training lifts counterexample success on incorrect programs from 40.61% to 62.67% and mean binary verdict accuracy from 71.41% to 87.41%. ARBITER also improves code selection and refinement, and in an IOI 2026 case study its guided refinement lifts GPT-OSS-120B's expected score above the gold-medal threshold under the official submission limit.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

ProAS (Progress-Aware Advantage Shaping) is a plug-and-play framework for multi-turn agentic RL that fixes the uniform credit assignment problem in critic-free algorithms like GRPO, where trajectory-level advantages are broadcast to every action. It uses a dynamically evolving Agentic Reference Graph with Masking Unreasonable Incentives and Asymmetric Penalty Redistribution to give finer-grained credit without a critic. On Qwen2.5-1.5B/7B-Instruct it reports success rates up to 91.9%/94.3% on ALFWorld and 66.7%/70.8% on WebShop, with up to 2.44x faster training convergence and robustness to imperfect or distilled ARGs.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

Legal-R1 is an agentic retrieval framework for evidence-based legal question answering that splits reasoning across three agents. A Lawyer agent decides when to retrieve, which legal source to query, what evidence to seek and when it has enough. A Legal Assistant retrieves statutes and precedent cases with authority-aware and diversity-aware selection, and a Judge agent writes the final answer from the full evidence trajectory. Only the Lawyer agent is post-trained, using masked supervised fine-tuning and trajectory-aware reinforcement learning. The paper reports gains over general-purpose, legal-specific and retrieval-augmented baselines on several Chinese legal benchmarks.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

Technical report for Xiaomi's MiMo-V2.6 omni-modal model family, which scales RL compute along three axes: asynchronous training with 1,568 samples and 2.7-3.7B tokens per step at up to 1M context, more diverse environments (code, general, visual, cyber) under mixed agent harnesses, and groupwise agentic grading that gives better reward signals and favors shorter, token-efficient solutions. Stability measures include a frozen MoE router and a multi-layer defense against reward hacking, built on a mid-trained hybrid-SWA architecture. The authors open-source training dynamics, RL environments and the RL framework, which makes it a useful reference for reproducing scaled agentic RL.

huggingface.copaperAdded 10 Oct 2026🧠 AI

AgentTune is a source-available Python library that unifies building, training, evaluating, distilling and maintaining tool-using LLM agents. Its core is a two-tier EventLog representation that preserves agent trajectories and yields consistent projections for training, evaluation and auditing. On top of it sit a shared agent runtime, tool-calling reinforcement learning, agentic distillation, RAG, decision workflows, evaluation utilities and a designed path toward self-healing agents, demonstrated on tasks such as legal contract search.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

Technical report for the MiMo-V2.6 omni-modal model series, which scales RL compute after mid-training on a multimodal corpus, using a hybrid sliding-window-attention architecture. RL is scaled along three axes: asynchronous training with 1,568 samples and 2.7-3.7B tokens per step at up to 1M context; diverse environments (code, general, visual, cyber) under mixed agent harnesses; and groupwise agentic grading that gives better rewards for long-horizon tasks and favors shorter, token-efficient solutions. Stability measures include freezing the MoE router and a multi-layer defense against reward hacking. The authors open-source the training dynamics, RL environments and RL framework.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

TRACE (Train-Rollout Quantization Alignment via Compact GuidancE) is an FP4 quantization framework for reinforcement learning post-training of Mixture-of-Experts language models. It uses rollout-guided quantization-aware training to align training-side FP4 rounding with rollout-side quantization outcomes, reducing the train-rollout discrepancy that limits prior low-precision RL methods, and adds a caching scheme that selectively retains mantissa/scale information from deeper layers to cut communication overhead. Evaluated on four large MoE models across reasoning, coding and long-horizon RL tasks, it matches BF16-rollout RL performance while achieving up to 5.4x rollout speedup with joint FP4 weight/activation and FP4 KV-cache quantization.

arxiv.orgpaperAdded 7 Oct 2026🧠 AI

MemPilot is a framework that lets LLM agents curate memory on demand rather than query-agnostically, using a multi-step policy trained via reinforcement learning to choose between retrieving from pre-built memory or delegating query-specific curation of raw multimodal history to heterogeneous LLMs and VLMs. The policy controls evidence amount, curation instructions, model choice, and visual access, and is optimized with objective-wise advantage decoupling and prefix-based marginal utility estimation to balance performance, cost, and latency. Tested across five multimodal agent-memory benchmarks, it achieves broader performance-cost-latency trade-off frontiers than existing trade-off-aware baselines.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

MedPrune is a multi-agent framework for medical visual question answering that models the diagnostic process as a heterogeneous communication graph of specialist agents across departments, then dynamically prunes both nodes (irrelevant specialist agents, via reinforcement learning) and edges (low-value intra-/inter-departmental connections) to cut redundant communication. The authors report it beats existing multi-agent baselines on medical VQA benchmarks under both full-set and few-shot settings while improving token efficiency and adversarial robustness.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

RubricArmor is an adversarial framework for generating evaluation rubrics used in rubric-based reinforcement learning of LLMs, designed to preempt reward hacking before RL training begins. It alternates an attack step, which synthesizes adversarial responses that satisfy the current rubric while failing the actual task, with a repair step that revises the rubric to close the exposed gaps while keeping valid criteria intact. The paper reports that rubrics produced this way outperform existing LLM-based rubric generation baselines and lead to more effective downstream rubric-based RL.

arxiv.orgpaperAdded 6 Oct 2026, published 4 Oct 2026🧠 AI

NeutronGym is an executable environment for neutron scientific-instrument design where LLM agents build instruments via validating tools, McStas ray-traces the result, and a level-resolved ladder grades syntax, runtime, structure and physics without an LLM judge. A curated McStasBench slice adds 16 tasks from published instruments; seven frontier models solve at most 7 of 16, but reinforcement learning on the ladder's reward takes Qwen3-8B from 11% to 77% on held-out instances (surpassing an untrained Qwen3-32B), with ablations showing the partial-credit ladder is responsible for most of the gain.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

VenusRL is a fully disaggregated system for agentic reinforcement learning that trains LLM agents through multi-turn interactions with external tool environments. It introduces a priority-aware action scheduler that uses length-prediction heuristics to prioritize sample groups likely to unblock the next training step across batch admission, KV cache residency, and cross-worker orchestration, plus an environment resource manager that packs more tool sandboxes per node via a memory-aware admission threshold and template-keyed page-sharing pool using copy-on-write for isolation. The system achieves up to 4.24x end-to-end training speedup and reduces environment (sandbox) cost by up to 89% over state-of-the-art baselines on agentic RL workloads.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

The paper introduces Selection-based Structured Reasoning (SSR), a framework that replaces free-form chain-of-thought generation in multimodal search agents with selection among pre-specified, reusable natural-language reasoning candidates, scored by likelihood via teacher-forced prefilling with a shared KV cache. Evaluated on seven multimodal search benchmarks using 2B and 4B scale models under both reinforcement learning and supervised fine-tuning, SSR matches the success rate of leading search agents of the same scale while cutting per-turn reasoning latency by over 90% and total per-question inference latency by 28-54%. It targets the problem of small models producing long, low-value reasoning traces before each action, which is costly at inference time.

arxiv.orgpaperAdded 2 Oct 2026, published 1 Oct 2026🧠 AI

LineupRL is an RLVR (reinforcement learning with verifiable rewards) pipeline for training time series captioning models, where the reward comes from a frozen LLM verifier that must match a generated caption back to its source time series among distractors, rather than judging caption quality directly. The authors show a 3B vision-language model trained with LineupRL outperforms SFT and RL baselines across two captioning benchmarks plus downstream forecasting/reconstruction tasks, and even beats the 72B VLM whose captions the SFT baseline distills from, using 1/24 the parameters, while resisting reward hacking.

arxiv.orgpaperAdded 2 Oct 2026, published 1 Oct 2026🧠 AI

This paper proposes Semifactual Credit-Augmented Policy Optimization (SCAPO), a variant of Group Relative Policy Optimization (GRPO) for reinforcement learning with verifiable rewards (RLVR). It uses semifactual prompt interventions to measure token-level probability drift and derive normalized stability scores, reducing advantage credit for unstable (likely spurious) tokens during early training rather than uniformly assigning the same advantage to every token in a response. On Qwen3-4B-Base and Qwen3-1.7B-Base, SCAPO improves AIME 2024-2026 accuracy over GRPO by 5.63 and 4.17 percentage points respectively, and achieves the best results on most math benchmarks and all out-of-distribution benchmarks tested.

arxiv.orgpaperAdded 1 Oct 2026, published 30 Sep 2026🧠 AI

UMM-Reflection is a reinforcement learning method that trains a unified multimodal model to jointly learn reflection text and image revision within a single closed loop: diagnose an image's flaws, revise it, observe the result, and repeat. Sibling trajectories sharing one initial image enable a group-relative advantage that compares whole reflection strategies rather than assigning credit per round, and one trajectory-level advantage updates both the reflection tokens and the flow-based revision head jointly, unlike SFT cold-starts or RL that optimizes only the renderer. On BAGEL, it improves GenEval by 12.05 points over SFT and transfers to unseen benchmarks WISE, OneIG-Bench and T2I-CompBench++, with no external verifier needed at inference.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

This paper proposes KV streams, a technique for efficient context compaction during agentic LLM reinforcement learning training, addressing the GPU memory bottleneck caused by ever longer context traces. Instead of flushing and re prefilling the KV cache after each compaction step, KV streams carries the KV cache forward like a recurrent state, yielding a 2.6 to 5x wall clock training speedup across three compaction strategies without hurting performance. The streamed KV cache also preserves information long after it leaves the visible context, and this recurrent like behavior emerges from RL training alone.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

Introduces "harness learning," a method that trains a proposer model via reinforcement learning to revise a solver agent's harness (the executable scaffolding coordinating model calls, tool use, and information flow) using execution feedback, treating harness revisions as analogous to weight updates in meta-learning. At test time the proposer refines the harness across successive executions on a new task without any parameter-space update, and experiments on reasoning and multi-hop QA show revision quality improves with training and that test-time adaptation transfers to unseen tasks. The work points toward continually learning agents that convert accumulated experience into generalizable harness improvements rather than model weight changes.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

This arXiv preprint introduces AI Night-Scientist, an agentic framework that uses GRPO reinforcement learning to counteract LLMs' low-entropy bias toward homogeneous outputs in open-ended scientific ideation. It models creativity along three axes (action, process, outcome) grounded in cognitive science, and trains models to depart from predictable reasoning via semantic guidance rather than just raising decoding temperature. Reported results show a 27.8% expansion in research-direction diversity, a 14.9% increase in contribution types, up to 32.0 points higher predicted citation impact, and 66.2 points higher originality versus the base model.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

This arXiv preprint argues that distillation defenses for closed-source LLMs are typically evaluated under an unrealistic threat model that assumes attackers stop after distillation, and shows that adding a reinforcement learning stage afterward breaks many existing defenses. The authors demonstrate that simple distillation attacks using data readily available from current model APIs, followed by RL, can reproduce reasoning improvements comparable to more sophisticated attacks that extract full hidden reasoning traces — meaning any defense that leaks enough information to reconstruct approximate traces is likely ineffective. They conclude by discussing batch-level distillation defenses as a potentially more robust alternative.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

This arXiv paper argues that existing defenses against distillation attacks on closed-source LLMs are evaluated under an unrealistic threat model that assumes no further training after distillation. The authors show that simple distillation attacks using data obtainable from current commercial APIs, followed by reinforcement learning, can match the reasoning gains of more sophisticated full-trace extraction attacks, breaking defenses that appeared effective when evaluated right after distillation. They conclude that any defense which leaks enough information to approximately reconstruct reasoning traces is likely ineffective, and suggest batch-level defenses as a more promising direction.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🛡️ Security

This arXiv paper analyzes reward hacking in reinforcement learning with verifiable rewards (RLVR), using gradient flow analysis with a fixed verifier to characterize when reward increases while correctness decreases. The authors prove that standard RLVR training signals are insufficient to detect or reduce "accepted errors" (incorrect responses rewarded by the verifier) without also suppressing correct responses, and propose a "selective control" correction that incorporates partial audit feedback on correctness to simultaneously lower accepted-error probability and raise correct-response probability. Experiments on log-linear and neural contextual bandits and a language model support the theory, showing selective control under partial auditing reduces accepted errors while increasing correctness.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

This paper introduces STL-SVPG (Signal Temporal Logic-guided Stein Variational Policy Gradient), a population-based reinforcement learning method that uses smooth STL robustness as a trajectory-level training objective instead of conventional dense reward shaping. By differentiating the STL objective through system dynamics, the method assigns credit to policy actions based on their effect on the full task specification — capturing event-triggered responses, strict ordering, deadlines, and physical contact — rather than local progress alone. Evaluated on six quadcopter and manipulator tasks, STL-SVPG achieves the highest success rate on five of six benchmarks and transfers temporal/contact behavior from simulation to real robots.

arxiv.orgpaperAdded 28 Sep 2026, published 25 Sep 2026🧠 AI

Introduces TRACE (Temporal Reconstruction Attack on Consecutive Encodings), an amortized gradient-inversion attack that reconstructs sequences of private observation-action trajectories from per-step policy-gradient updates shared by embodied RL agents in distributed training. The attack exploits cross-time correlation between successive gradients and a closed-form recovery of actions from policy-head gradient structure, reaching 18.8 dB PSNR with near-perfect action recovery in 3-4.5ms per frame, outperforming prior single-frame and optimization-based gradient-inversion baselines across recurrent, residual, and transformer architectures. The authors argue defending against it requires sequence-aware privacy mechanisms rather than single-step gradient protections.

arxiv.orgpaperAdded 26 Sep 2026, published 24 Sep 2026🛡️ Security

This paper presents a distill-then-RL framework for query understanding (QU) in Roblox's game search, where an LLM is first supervised fine-tuned via teacher-student distillation to produce schema-compliant outputs (intent classification, query expansion, etc.), then optimized with reinforcement learning using component-specific rewards derived from live interaction with the search engine rather than a single end-to-end reward. On Roblox search, this component-specific RL raises NDCG@20 by 8.9 points over the SFT baseline and 3.5 points over end-to-end reward training, showing that decomposed, search-aware rewards better capture how each QU component affects downstream retrieval and ranking.

arxiv.orgpaperAdded 26 Sep 2026, published 24 Sep 2026🧠 AI

VideoGen-Agent is a multimodal agent trained with multitask agentic reinforcement learning to coordinate augmentation, generation, and verification tools for video generation, using supervised fine-tuning on teacher trajectories followed by RL with a category-aware hybrid reward. The authors introduce VABench, a 600-prompt held-out benchmark covering procedural knowledge, identity preservation, physical consistency, scene composition, and multi-shot temporal structure, on which the agent improves over its base text-to-video model from 56.5 to 75.6 (and 86.1 when upgrading the underlying generation tools).

arxiv.orgpaperAdded 22 Sep 2026, published 21 Sep 2026🧠 AI

PISmith trains an attack LLM with GRPO in a black-box setting to generate injected prompts that defeat prompt-injection defenses. It addresses extreme reward sparsity, which causes entropy collapse in standard GRPO, with adaptive entropy regularization and dynamic advantage weighting. Evaluated on 13 benchmarks against 8 defenses and 7 baselines (static, search-based such as TAP and PAIR, and RL-Hammer), it achieves the highest attack success rates and also works on InjecAgent and AgentDojo against GPT-4o-mini and GPT-5-nano. It shows state-of-the-art defenses such as Meta-SecAlign remain vulnerable to adaptive attacks.

arxiv.orgpaperAdded 17 Sep 2026🛡️ Security