280 results for “Lua”

GPT-Rosalind-6 System Card Addendum is OpenAI's safety documentation for a life-sciences model that shares weights with GPT-6 Astra. It rates the model High in Biological and Chemical and Critical in Cybersecurity under the Preparedness Framework. It reports biology evaluations, trusted-access controls, biosecurity refusal pass rates, a historical cyber jailbreak regression (14.3% task success), and sabotage and monitor-evasion results.

deploymentsafety.openai.compaperPublished and added 30 Oct 2026🛡️ Security

GPT-Rosalind-6 System Card Addendum describes OpenAI's life-sciences reasoning model, which uses the same weights as GPT-6 Astra and is offered only through trusted access. It reports life-science benchmark gains over GPT-Rosalind-5.5 (MedChemBench v7 49.6% vs 28.9%), High-capability Biological and Chemical Preparedness results, and below-threshold Critical bio design evaluations. It also covers access controls, refusal evaluations, cyber jailbreak regression and alignment tables.

deploymentsafety.openai.compaperPublished and added 30 Oct 2026🧠 AI

Glance is a RAG method in which the generator also reranks. It adds a lightweight decision adapter and pointer head that score candidate passages from one shared prefill, so a single resident 9B backbone does generation and reranking. After 39 minutes of MS MARCO fine-tuning it reaches nDCG@10 of 0.741 on SciFact and 0.345 on NFCorpus, against 0.721 and 0.326 for bge-reranker-v2-m3. On 300 HotpotQA questions it improves answer F1 by 0.045 over cross-encoder-selected context. Evaluation covers two retrieval datasets, one QA dataset and one backbone family.

arxiv.orgpaperAdded 31 Oct 2026, published 29 Oct 2026🧠 AI

AutoBench-CT is a modular benchmark for constant-time analysis tools, with manually verified per-violation ground truth across 64 cryptographic tasks. It scores seven tools on operational performance, detection quality and usability. The authors explain why CT-grind (2010) is still the most used tool, list recurring limitations across tools, and propose a composite pipeline for stronger constant-time guarantees. Accepted at ACSAC 2026.

eprint.iacr.orgpaperAdded 31 Oct 2026, published 29 Oct 2026🛡️ Security

TrussTutor-Bench is a benchmark of 200 verified two-dimensional truss problems from statics, paired with automatically generated student solutions. It evaluates thirteen LLMs on solution accuracy (64.5% to 97.0%), error detection (90.4% to 100%) and sycophancy. Models rejected their own correct solutions in 1.6% to 50.0% of cases when a student insisted the work was wrong. Sycophancy separated models; error detection did not.

arxiv.orgpaperAdded 31 Oct 2026, published 28 Oct 2026🧠 AI

QbD-VLA is a mixed-precision post-training quantization method for vision-language-action models. It uses task success from short closed-loop evaluations to assign weight and activation bit widths, accounting for interactions between components. On four VLA models across all four LIBERO suites it holds task performance at about 4-bit weights and under 6-bit activations. The gain on SIMPLER is small. On a Franka Research 3 arm, X-VLA shows no detected loss, but π0.5 fails.

arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026🧠 AI

Repairs for Whom? studies single-step repairs of failed LLM agent trajectories and asks whether common evaluators (greedy replay, stronger models, oracles, log-reading judges) pick repairs that help the deployed agent. Using independent rollouts in ALFWorld and WebShop, greedy replay loses 0.06-0.13 in expected gain, and Qwen2.5-14B rollouts misselect repairs for Qwen2.5-7B. The authors recommend verifying repairs with the executor that will run them.

arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026🧠 AI

The paper proposes POB-ICM (contingency matrix with Chi-square independence testing) and POB-UNBP (a uniformity null model under permutation-based evaluation) to detect positional bias in top-1, single-turn LLM recommenders. On a RAG product-recommendation testbed with 2000 queries and 20 permutations each (40,000 instances), both methods reject positional independence in all evaluated models. Three metrics quantify the effect, and the bias persists across candidate-set size and temperature. Code is public.

arxiv.orgpaperAdded 30 Oct 2026, published 27 Oct 2026🧠 AI

Heads of Security (HoS) is a prompt injection defense that enforces instruction-data separation with learned per-head attention gates, redistributing attention between trusted and untrusted tokens. Evaluated on Llama-3.3-70B-Instruct and GPT-OSS-20B against SecAlign++ and ASIDE. The black-box RL-Hammer attack reaches 97% ASR on GPT-OSS-20B and 85% on SecAlign but 10% on HoS. The white-box Claudini attack needs 3 times more FLOPs on HoS. HoS keeps base-model utility in few-shot settings, where SecAlign and ASIDE degrade.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🛡️ Security

This paper presents an annotated egocentric tennis dataset (egocentric video synced with five allocentric views, global positions, contact times, stroke types) and evaluates image encoders, video encoders and vision-language models on seven prediction tasks. Best models reach 0.14 m player and 0.54 m ball localization error and 0.14 s contact-time error. Depth input helps 3D ball localization, and longer histories do not help forecasting. A conditional diffusion model generates whole-body tennis motion in real time.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

OpenGray is a benchmark that adds tasks with known correct decisions to the OpenKBP-Opt head-and-neck dataset, including two overlap contradictions with provably incompatible dose goals and a feasible control. Four planning systems, from a rule-based script to a full LLM agent, were tested on 30 patients with Terra and Ministral 3 8B. Decision accuracy reached 95.4% and 92.1%, yet only 50% and 25% of overlap contradictions were escalated. SAGE logs of 39 radiosurgery runs showed an agent that lacked an action for stopping.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

p-DCIM is a 28 nm SRAM digital compute-in-memory macro (0.16 mm2, 0.41 mW) that works as an Ising machine for RAG evidence selection. Selection is cast as a QUBO problem, and the same MAC datapath computes embedding similarities and annealing steps. Replacing LLM reranking with it gives a selection-energy estimate over six orders of magnitude lower and cuts total energy per query by 67%, with comparable token-level F1 on the evaluated examples.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

AlgOssa is a decentralized secure aggregation protocol built on Ossa, a generic committee-aided, single-server construction for one-shot clients (following OPA and Willow, CRYPTO 2025). Ossa realizes secure aggregation with non-selective abort against a malicious server. AlgOssa adds Algorand-style sortition to rotate leaders and committees, bounds the chance of an honest server, and gives eventual delivery. Classical and post-quantum routes are evaluated.

eprint.iacr.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🛡️ Security

Poly2MsgLeaks is a profiled chosen-ciphertext power-analysis attack on the SMAUG-T (TiMER parameter set) post-quantum KEM that recovers the full 512-coefficient PKE secret from six traces on an STM32F415. It recovers 999 of 1,000 unseen secrets and proves an incorrect key is released with probability at most 2^-2095.6. Single-write message assembly and a two-share masked P2M resisted the evaluated six-execution attack.

eprint.iacr.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🛡️ Security

Log-To-Leak is a prompt-injection attack on tool-using LLM agents that abuses Model Context Protocol tool metadata to trigger an attacker-controlled logging call after the last legitimate call, leaking the user query, server name, server response and agent response. Evaluated on five MCP servers and four agents, it reaches 62.64-100% attack success with 76.55-95.64% logging accuracy. Three defenses leave over 55% residual ASR on GitHub MCP with GPT-4o.

arxiv.orgpaperAdded 28 Oct 2026, published 26 Oct 2026🛡️ Security

Scope-Gap Injection (SGI) is an indirect prompt injection method for multi-stage LLM-agent safety pipelines, tested in the IJCAI 2026 OpenClaw Agent Security Attack Competition. The visible task and skill stay benign while target identifiers, bypass parameters and authorization steps sit in an attachment written as a business worklist, which the S2 LLM judge appears not to inspect. SGI bypassed all three stages (static scan, LLM judge, live audit) for 5 of 19 targets, best F = 0.385. Attachment placement had 78% S2 acceptance versus 23% in-skill. The paper closes with defenses that align evaluator scope with agent channels.

arxiv.orgpaperAdded 27 Oct 2026, published 26 Oct 2026🛡️ Security

This paper evaluates sycophancy in medical diagnosis across nine AI models. Each model is shown clinical evidence plus a patient's belief in a specific but incorrect condition, with the belief's closeness to the evidence varied. Models adopted the belief 23.2% of the time on average, even for fictional conditions, and 43.0% under a Rare Disease Claimer persona. Medical fine-tuning and larger scale reduced susceptibility.

arxiv.orgpaperAdded 27 Oct 2026, published 25 Oct 2026🧠 AI

Continual Learning Exam (CLE) is a benchmark of 120 human-authored tasks, each a five-level ladder (600 levels) across 89 professional roles and 12 occupational domains, built by 50+ domain experts. Agents must pass a level to unlock the next, and an evaluation agent scores submissions against human rubrics. Across 17 harness-model configurations, feedback plus one revision adds a median 5.3 points; the best system clears 47.7% of levels and finishes all five levels in 23.3% of tasks.

arxiv.orgpaperAdded 27 Oct 2026, published 25 Oct 2026🧠 AI

BARBICAN is a fail-closed prompt-injection gateway for MCP tool mediation. This paper shows its arbitration classifier ran on tokenizer-mangled text for a whole evaluation. Fixing it raised AUC from 0.392 to 0.855 while system recall fell from 80.67% to 80.34%. The routed band is 96.82% malicious, so deletion ablations mislead without substitution experiments and class-balance reporting.

arxiv.orgpaperAdded 27 Oct 2026, published 23 Oct 2026🛡️ Security

The paper analyses BARBICAN, a fail-closed prompt-injection gateway for MCP tool mediation, using deletion, substitution, repair and transplant experiments. Its arbitration classifier ran on tokenizer-mangled text for a whole evaluation. Fixing that raised AUC from 0.392 to 0.855 while system recall fell from 80.67% to 80.34%. The band it receives is 96.82% malicious. The authors argue ablation figures need the band's class balance and substitution baselines.

arxiv.orgpaperAdded 27 Oct 2026, published 23 Oct 2026🧠 AI

NTBench (Near-Target Benchmark) tests LLM and agent robustness to near-target prompt injection, where injected instructions steer a response toward a plausible but incorrect or harmful outcome instead of abandoning the task. It builds reference-guided target-instruction pairs on existing datasets, with human validation, and evaluates seven LLMs on ten language datasets and two agent benchmarks. Attack success rose 9.3 to 35.8 points (incorrect targets) and 1.7 to 17.3 points (harmful targets) over fixed-target baselines.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🛡️ Security

When Agents Fail: Model or Harness--Who's to Blame? proposes five measures for evaluating agent harnesses together with their underlying models: Valid Completion Rate, Model-Harness Fit, Cross-Model Fit, Interaction Residuals and Token Load. It tests four harnesses (OpenCode, Codex CLI, Pi CLI, DeepAgents) with four models on six benchmarks, across 111 settings and 52,168 trajectories. Harness rankings change by model and benchmark. OpenCode ranks first in cross-model performance.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

Tampered ML is a benchmark of 1,858 item-pairs, each pairing an experiment proposal reconstructed from a published ML paper with a minimally edited version that adds a targeted methodological flaw. Models must pick the sounder proposal in both presentation orders. A human study estimates 86% of items are valid (95% CI 81-91%). Opus 5 reaches 77.9% accuracy, and models are sensitive to order and favor flawed designs that look more specific, transparent, or conventional.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

This paper presents an offline evaluation harness for a production merchant-facing conversational agent. It stays reproducible when live backend data changes and can be regenerated when it goes stale. Used as the fitness function for GEPA prompt optimization under a correctness gate against reward hacking, it shrinks the full prompt surface by 3.5-7.6% without losing held-out correctness, across three subject-model families.

arxiv.orgpaperAdded 26 Oct 2026, published 22 Oct 2026🧠 AI

EpiSEM-Bench is an executable benchmark of 300 matched unsafe/safe task pairs across six domains. It tests whether tool-using LLM agents take prohibited actions when a tool response omits a safety-relevant fact, even though the request is benign and all returned statements are true. It finds large variation across ten models, and recovery access cuts unsafe execution for all ten.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

CausalCoT is an inference framework that splits a problem into a directed acyclic graph of atomic sub-questions and evaluates each node in an isolated context holding only its direct causal parents. This removes attention paths to non-parent content, so intermediate steps drive the answer rather than leaking from global context. It keeps benchmark accuracy competitive and raises causal sensitivity of reasoning steps by 12 points on average, up to 21, over the strongest baseline. The graph also allows test-time corrections that propagate downstream.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

This paper describes an evidence-aware retrieval evaluation protocol for a clinical RAG system over breast lesion management literature. Relevance is defined by evidence spans mapped to all containing chunks. A hybrid retriever (dense, BM25, cross-encoder reranking) with 384-token chunks and 20% overlap reached 0.94 Recall@3 on 18 queries. Expanding ground truth from 25 to 37 spans did not change rankings. The authors note the small set needs independent validation.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

This empirical study benchmarks recent LLMs against fine-tuned encoder-decoder models on knowledge graph-to-text generation, using reference-based metrics and graph-grounded, reference-free LLM-based evaluation validated against human annotations. Recent LLMs narrow or close the gap on reference-based metrics and do better on graph-grounded ones. Prompted models make fewer contradictions and triple omissions, and fine-tuned models' errors grow faster with graph size.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

This October 2026 arXiv paper evaluates four LLM inference optimization families on Apple Silicon: KV-cache pruning, speculative decoding, ported GPU kernels and post-training quantization. Tests run on four Apple machines with a single-request decode workload, plus matched RTX 4090 runs. KV-cache pruning reaches 1.75x throughput on Apple, all tested speculative decoding methods reduce throughput, and ported kernels give up to 1.20x.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

Thomas Decru's ePrint 2026/2595 proves a corrected version of radical isogeny formulae for every N≥4, odd and even, over fields of characteristic coprime to 2N. It settles the sign choice for even N so that 2-isogeny walks stay on the volcano surface. One radical step costs about 12.5N multiplications and one exponentiation, and the paper shows how to evaluate radical isogenies on points, so a CSIDH class group action needs no torsion point sampling.

eprint.iacr.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🛡️ Security

The SC-matrix is an evaluation framework for multilingual LLM sycophancy. It crosses a model's competence (correct answer with no pressure) with its sycophancy (keeping that answer under pressure), so agreeing with a false premise can be told apart from not knowing the answer in that language. Results: equal aggregate sycophancy rates hide different failure mixes, high competence does not always protect, and reasoning-oriented mitigations beat directive ones.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

IFC-MCQ is a method that automatically generates multiple-choice benchmarks from Industry Foundation Classes (IFC) building-model files, so answers can be graded by exact option-set match instead of human or LLM review. Its taxonomy has 15 categories and 36 subtypes in four groups, with answer keys from the file and distractors from file values or rules. Two agentic setups, tool-based and code-based retrieval, showed category-specific failures and variation by IFC model and interface.

arxiv.orgpaperAdded 23 Oct 2026, published 21 Oct 2026🧠 AI

SIM-UT (Simulation of User Behaviour for Triage) is a multi-turn, simulation-based method for finding failure modes in LLM medical triage chatbots. It was applied to three WhatsApp health chatbots (India, South Africa, Kenya) across 5,000+ multilingual simulated turns, comparing MedGemma with ChatGPT and Claude. The main finding is de-escalation risk: agents that first flagged emergencies agreed with users who minimised symptoms, such as vaginal bleeding in pregnancy.

arxiv.orgpaperAdded 24 Oct 2026, published 21 Oct 2026🧠 AI

Sotto Maschera is a masked implementation of the Sota Voce sparse fixed-weight vector sampler (Sota support sampler and MWSupport conversion) for embedded devices, protecting against power side-channel attacks. It bitslices sorting, merging and compaction and masks only in the Boolean domain, avoiding slow conversions. TVLA evaluations show no leakage, and HQC runs up to 45x faster than previous masked work.

eprint.iacr.orgpaperAdded 24 Oct 2026, published 21 Oct 2026🛡️ Security

Protection Is Not One Event is a survey and human-coded evidence map of 118 RAG defense papers, grouped by measured endpoint into a seven-family taxonomy (kappa=0.867 across 32 annotators). It finds that only 17.8% of studies use content disclosure as their primary endpoint, and that most evaluations use general QA data and non-adaptive attacks. In a testbed on Qwen3-8B, an 8-gram guard cut copying to 0.0%, yet a three-query adaptive attacker still recovered 48.3% of hidden facts. Taxonomy, dataset, coding protocols and testbed are released.

arxiv.orgpaperAdded 24 Oct 2026, published 21 Oct 2026🧠 AI

DIVE is a benchmark for full-duplex interaction with visual engagement in continuous audio/video streams. It has 525 human-annotated samples across six tasks in three scenarios: proactive intervention, continuous engagement and interaction coordination. Evaluated real-time models reach sub-second average latency, but window response rates stay at or below 53.77%, and adapting ongoing responses to new information is hardest.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

Systematic evaluation of fault attacks on ML-DSA (FIPS 204) rejection sampling and of proposed countermeasures. It proves the "third rejection test" of patent WO2025261784A1 re-verifies what check 2 already guarantees, and shows that 30,000 fault injections never detect a skipped z-norm check. The author fully recovers s1 against a third-check-protected implementation under randomized signing. Third check and sign-then-verify are complementary; overhead is 0.04–0.05% on x86-64 and 0.83% on Cortex-M4 for the third check.

eprint.iacr.orgpaperAdded 21 Oct 2026, published 20 Oct 2026🛡️ Security

Cost-Coupled Gittins (CC-Gittins) is a multi-objective extension of cost-aware Gittins search that recommends Pareto-optimal model-to-role assignments for LLM agentic workflows. It exploits the coupling between evaluation and deployment cost, alternating between seeking higher quality and lower cost. On eight quality-cost matrices (two to five roles, up to 1,681 assignments), it recovers a substantial share of the true Pareto frontier at about 10% of exhaustive-evaluation cost, and most of it by 30%. Latency is tested as a third objective.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

This arXiv paper tests eight jailbreak attacks against three open-weight language models under three evaluation procedures. Low-resource language attacks score 96.0% mean ASR under keyword matching but 2.7% under a Qwen3 judge using StrongREJECT instructions. At Ollama's default temperature of 0.8, 100 samples of an unchanged request give 100% ASR by keywords and 50% by the judge. The authors recommend explicit success definitions, matched response budgets, validated evaluators and cumulative-failure reporting.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🛡️ Security

Concerns in Benchmarking Jailbreaks is an October 2026 arXiv study of eight jailbreak attacks on three open-weight models under three evaluation procedures. Low-resource language attacks score 96.0% ASR by keyword matching but 2.7% under a Qwen3 evaluator using StrongREJECT instructions. Attack rankings flip with attempt budgets, and 100 samples at Ollama's default temperature 0.8 give 100% ASR by keywords, 50% by the evaluator.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

This paper audits the PathQuestion (PQ) and PathQuestion-Large (PQL) multi-hop knowledge-graph QA benchmarks as graph-navigation tasks. It holds questions, answers, path annotations and splits fixed, and replaces each released task-filtered graph with the larger source KG. Restoration cuts released-answer accuracy and trajectory fidelity for frozen small-language-model policies and trained MINERVA policies, and graph-consistent evaluation recovers only part of the loss.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

Concrete LDVR Attacks with Fewer Signers is an ePrint paper by Renas Bacho on the low-dimensional vector representation (LDVR) problem behind adaptive security of threshold Schnorr. It combines a Chinese remainder theorem construction with a generalized birthday search to attack 488 signers with 173 corruptions over the Curve25519 scalar field, down from 5,020 signers. For adversarial evaluation points, it reaches optimal success probability and breaks 228 signers with 114 corruptions in 168.60 seconds.

eprint.iacr.orgpaperAdded 20 Oct 2026, published 19 Oct 2026🛡️ Security

This paper is a controlled, large-scale evaluation of biomedical Retrieval-Augmented Generation across growing corpus sizes. It compares abstract, flat full-text and hierarchical full-text representations, with and without reranking, on multiple-choice QA over five external benchmarks and a held-out test set. RAG gave mean gains of up to 9.6 points externally and 10.2-30.3 points on the held-out set. Hierarchical indexing used 67% fewer units and 44-56% less storage than flat full-text.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

Privacy with Leakage from Falsifiable Assumptions is an ASIACRYPT 2026 paper by Francesco Berti and Itamar Levi. It replaces simulatability with "secure refreshing", a key-refreshing assumption that evaluation laboratories can test directly, and adds "strongly protected secure refreshing". Using these, it proves privacy of the PSV encryption scheme without assuming a leak-free component.

eprint.iacr.orgpaperAdded 20 Oct 2026, published 19 Oct 2026🛡️ Security

Self-evolving Agent Harness is a 66-page survey and theory paper on revising an agent's scaffolding (prompts, memory, workflows, tools, code, plugins) while model weights stay fixed. It frames search as evolutionary optimization and uses continual PAC-learning to derive a generalization bound, a retention bound and a high-probability admission rule with fresh validation data. It separates selection, activation, attribution and improvement as evaluation claims, and covers applications and governance. Code is at github.com/zgcagi/Harness-RSI.

huggingface.copaperAdded 19 Oct 2026🧠 AI

OpenGCUA, from Alibaba's MAI-UI team, is an open stack for generalist computer-use agents (GCUAs), where the model chooses how to observe, act and organize work across GUI, accessibility-tree, API and shell interfaces. Its design combines code execution (GUI actions written as programs with loops and branches) with harness-to-model decision transfer. On OSWorld-v2 it raises mean partial score by +19.6 points across eight models, including GPT-6 Astra and Claude Opus 5, and cuts reported steps per task by 53%. The release includes the harness, prompts, evaluation trajectories, a macOS real-device harness, a cross-device SDK and an interactive lab. It also shows large efficiency gaps between models: Qwen3.8-Max used 19.4x the reasoning tokens of GPT-6 Astra on real macOS tasks.

huggingface.copaperAdded 17 Oct 2026🧠 AI

This paper builds a benchmark of 500 debatable questions across ten domains to measure how user narratives steer LLM agreement. Each question has 11 matched conditions (one neutral, five mirrored narrative levels per opposing stance), giving 5,500 instances across seven LLMs. Causal framing and worldview synthesis raise agreement, personal experience can lower it, and GPT-5.1 ablations show bullet formatting also increases agreement.

arxiv.orgpaperAdded 20 Oct 2026, published 16 Oct 2026🧠 AI

MobileInject is a benchmark for testing MLLM-based smartphone agents against indirect multimodal injection. It has an interactive evaluation framework with 11 scenarios, 9 security-risk categories and 108 executable risk instances. It adds Cross-Modal Consistency Injection, Multi-Role Collaborative Deception, Reasoning-Camouflage Injection and a Reasoning Security Diagnosis Framework. Experiments find broad vulnerability, with many failures from early reasoning hijacking.

arxiv.orgpaperAdded 20 Oct 2026, published 16 Oct 2026🛡️ Security

Describes a general-purpose LLM-based AI tutor for computing courses that accepts questions in Mandarin and replies bilingually, also generating glossaries of technical terms. It uses a specialized multi-source RAG pipeline with a novel question extraction mechanism to draw on diverse course materials. Evaluated on 91 Piazza posts: 83.52% answered correctly and helpfully, 76.92% of questions got high-quality translations, and 64.47% got glossaries.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

Proposes a modified Physics-informed DeepONet (PI-DeepONet) architecture that removes redundancy in how the Branch network processes input functions across collocation points. It is mathematically equivalent to the original but uses smaller input tensors and fewer Branch evaluations. The authors derive training-time scaling laws, then test on 1D, 2D and 3D linear advection problems. Accuracy matches the original, the maximum speed-up is 3.75× in 2D, and a 3D model trains on an NVIDIA A100 only with the new formulation, because the original runs out of memory.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

PROBE (Policy Rule Observation via Behavioral Exploration) trains LLM-based auditors that act as realistic users and probe deployed tool-using agents for policy violations (pricing, privacy, confirmation, safety) over multi-turn dialogue. It formalizes auditing as a two-player Markov game with a composite reward balancing violation discovery, behavioral plausibility, conversation progress and policy diversity, and compares GRPO, DPO and GFlowNet optimization in a controlled setting. Trained auditors find broader, higher-yield violations than same-architecture zero-shot adversaries, even when much smaller than the target agent. This positions multi-turn behavioral auditing as a distinct evaluation axis alongside capability and safety benchmarks.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

Introduces P^3IBench, a benchmark for proactive video understanding in real-life streaming scenarios, covering three capabilities: daily perception, risk prediction and intervention. It evaluates 12 open-source and proprietary video-language models (streaming and adapted non-streaming) and finds they struggle to make well-calibrated decisions about when to respond. The authors also build extra proactive training data and train P^3I-Assistant-4B, which improves several tasks unevenly, showing that proactive assistance needs response timing and restraint as well as visual understanding.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

Shows a protocol-level reaction attack on CKKS homomorphic-encryption deployments that publish coefficient-to-slot (C2S) evaluation keys and let the client return a 1-bit validity response. A malicious evaluator shifts a public encryption key, uses the public C2S layer to repack it into a ciphertext whose validity depends on one secret coefficient, and recovers a ternary secret key with O(H) queries, within a factor 2/log2(3) of the entropy lower bound. Each query is re-randomized with a fresh encryption of zero to evade trivial detection. The authors implemented it end to end in Lattigo v6.2.0 and recovered full keys across four standard CKKS parameter sets. It does not break RLWE or CKKS IND-CPA security.

eprint.iacr.orgpaperPublished and added 15 Oct 2026🛡️ Security

Paper arguing that agent human-approval prompts count as a second factor only if agent compromise cannot forge them and they name exactly what is approved. It defines twelve attacks on the approval path (decoy recipients, amount cancellation, card padding, label spoofing, enrollment squatting, threshold splitting) and tests them against AP2, Microsoft's action-bound approval design and MCP elicitation. An audit of ten agent runtimes found no third-party verifiable approval record and post-approval argument changes in four of eight. The reference defense ASU (Agent Step-Up) renders the card from a canonical effect descriptor, has a phone passkey sign its digest, and executes only the stored call. It is proven in Tamarin (53 lemmas) and evaluated on 27,108 scripted trials and 609 AgentDojo pairs. The trusted card cut forged-narration attack success from 0.54 to 0.03 and raised LLM-approver catch rate from 0.30 to 0.98.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🛡️ Security

Evaluates a native probabilistic decision model (Jev), Llama Prompt Guard 2 (22M and 86M) and an LLM judge on 47,611 held-out examples covering direct/indirect prompt injection, hard-benign inputs, obfuscation and distribution shift. The LLM judge has the best AUROC (0.897), followed by Jev (0.871), Prompt Guard 2 86M (0.757) and 22M (0.714). Jev's native probabilities do not consistently beat post-hoc calibrated classifier scores, and temperature scaling improves calibration but not discrimination. At a 1% false-negative-rate target, every system needs over 91.8% escalation, so probabilistic outputs alone do not give reliable security decisions under distribution shift.

arxiv.orgpaperAdded 17 Oct 2026, published 14 Oct 2026🛡️ Security

ePrint paper evaluating ASCA (SAT-based) and SASCA (belief propagation) side-channel attacks on the AES-based MILENAGE authentication algorithm in USIM cards, using neural-network leakage predictions to recover subscriber key K and operator secret OPc from a single trace. SASCA alone succeeds in 87.7% of attacks but converges to a wrong OPc in 6.1%; ASCA succeeds in only 34% within a 1-hour timeout. The hybrid, where BP refines the predictions before they go to the SAT solver, reaches 93.8% with lower average runtime. It also proposes a countermeasure that masks each AES round key with an independent per-device secret, and the authors report it defeated the hybrid attack in all evaluated cases.

eprint.iacr.orgpaperAdded 15 Oct 2026, published 14 Oct 2026🛡️ Security

Proposes a cost model and metric for an adversary who collects side-channel traces classically and finishes the key search on a quantum computer. Shows any quantum algorithm needs Omega(sqrt(r)) cipher evaluations for key rank r, and that the quadratic advantage holds only up to a rank bound set by the NIST MAXDEPTH budget (about 2^58.6 for AES-128 at D=2^40). Defines a quantum-aware key rank computable from existing rank estimators, with simulated-leakage experiments showing a depth-limited quantum adversary is worth about 22 traces at SNR 0.125.

arxiv.orgpaperAdded 16 Oct 2026, published 14 Oct 2026🛡️ Security

Proposes RTL-level architectural changes to general-purpose processors that remove the root causes of micro-architectural leakage, so masked cryptographic software runs securely under the Robust but Relaxed (RR) d-probing model without security-aware compilers or hardware-specific software workarounds. Each countermeasure is formally proven secure and integrated into the Ibex RISC-V core, which is evaluated by tool-assisted leakage assessment of the full netlist and by physical measurements on an FPGA.

eprint.iacr.orgpaperPublished and added 14 Oct 2026🛡️ Security

Systematization of knowledge on private Transformer inference (PTI) that builds an empirical graph of 62 systems, with system attributes as node features and evaluation compatibility as edges. Node-level attribution finds that cryptographic progress drives most speedups but always co-occurs with confounders such as model fidelity, hardware and cost accounting. Edge-level auditing finds only 1 of 1891 candidate system pairs directly comparable. Graph-level analysis shows the scale frontier plateaued at 13B parameters and projects the cost of deploying current PTI techniques on production-scale Transformers.

eprint.iacr.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

MiniDog is a framework for training a text-to-image flow-matching diffusion transformer from scratch on four RTX 3090 GPUs, with 55 minutes of pretraining and 15 minutes of fine-tuning. It pretrains on 26k ImageNet dog images that were captioned, then fine-tunes on 2k FLUX.1-schnell generated images. Image and text features are precomputed and cached. The model is a 22M-parameter transformer with the E2E-INVAE semantic latent space, a small Qwen text encoder and in-context conditioning, and training is accelerated with iREPA and an EUPE perception encoder. The repo evaluates with FID, HPSv2, PickScore and VAE reconstruction metrics. It lowers the barrier to hands-on diffusion training research.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

OmniCompass is a manually annotated benchmark for video grounding and understanding with two tracks: OmniCompass-VL (820 videos, 2,460 instances; vision-language models) and OmniCompass-AV (240 videos, 720 instances; joint audio-visual evidence for omni-modal models). Each track covers temporal grounding, spatial grounding and QA across short/medium/long videos, and each instance is run in paired text-only and text+image target specifications. Evaluating 50 MLLMs shows open-source models lag proprietary ones, especially on audio-visual understanding, that precise temporal/spatial grounding remains hard, and that predictions are inconsistent across the paired specifications.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

Proposes Geometric Calibration of supervised fine-tuning and SPACE (Subspace Projection for Adapted Checkpoint Enhancement). SPACE is a post-hoc, data-free, training-free method that uses the pre- and post-SFT checkpoints to revert SFT update components lying in the principal subspace of the pre-SFT weights and keep those in the orthogonal complement. It is evaluated on Qwen2.5, Qwen3, Qwen3.5, Llama-3.1 and Olmo-3 models (2B–14B) with math, coding and medical SFT. It reduces out-of-domain regression and improves in-domain performance, for example +2.85 ID / +2.26 OOD on Qwen2.5-7B math SFT.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

Introduces Structured Recipe Optimization (SRO), a paradigm for automated LLM post-training that optimizes over a task-adapted recipe space covering data sources, data processing, cross-domain composition and training configuration. It pairs an LLM-based Recipe-Space Designer, which builds the recipe space for the target tasks, with a Multi-Domain Recipe Optimizer that uses structured trial evidence to guide LLM candidate construction and multi-objective Bayesian trial selection. SRO gets the highest average score in all evaluated target-model and agent-LLM combinations; for example, 41.1 vs 15.4 for the strongest baseline on Qwen3-1.7B with GPT-5.4, and 53.8 vs 40.8 on Qwen3-4B with DeepSeek-V4-Flash, in joint post-training across three capability domains and six benchmarks.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

Short paper analysing the HM-QCT hybrid quantum–computational time-lock QKD protocol (Mazzoncini et al., 2025). It spells out the two "harvest now, decrypt later" strategies (quantum memory and classical memory) and shows each is excluded by exactly one of the two assumptions: short-term encryption and low-fidelity quantum storage. It then proposes HM-PRF, which replaces the short-term encryption with a locally evaluated pseudorandom function, so security needs only one-way functions rather than a predicted lasting gap in quantum memory technology.

eprint.iacr.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🛡️ Security

EmoGenie is a benchmark for emotions expressed and evoked by text-to-image outputs. It has 6,400 images from eight T2I models using 800 emotion-targeted prompts. It includes EmoGenie-QA (90,000 closed-set and 1,057 open-ended questions) and EmoGenie-Assess (13,175 mean opinion scores for valence, arousal, dominance and aesthetic quality). The authors also propose EmoProbe, which pairs emotion QA with response-aligned, dimension-specific regression heads. They report that it beats baselines on emotion QA and correlates better with human ratings. The work shows how reliably MLLMs interpret affect in generated images.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

The paper introduces verifiable reconstructible straightline codes (VRSCs) as an abstraction for compiling zkSNARKs into Universally Composable (UC) secure protocols. It shows that the witness-succinct UC compiler of Ganesh et al. (EUROCRYPT 2023) is missing a security requirement: permuting previously revealed evaluation openings yields a fresh accepting proof, which invalidates the claimed simulation argument. The fix uses verifier-enforced disjoint evaluation domains and a pairing-based Fischlin-style instantiation with structured Reed-Solomon codes and jointly computed KZG openings. This brings certificate generation to O(n log n) with constant extra proof size and verification work.

eprint.iacr.orgpaperAdded 14 Oct 2026, published 12 Oct 2026🛡️ Security

Paper presenting the first on-chip, TDC-sensor-based power side-channel attack on a co-tenant FPGA to reconstruct full-scale color ImageNet inputs (224x224) from a victim's first CNN convolution layer, using ML-based upscaling and recoloring. It separates genuinely leaked information from ML-generated post-processing. It also evaluates a pixel-substitution defense based on Hamming Weight, which is largely defeated by an attacker retrained on the transformed data. The authors conclude that robust defenses need randomized or input-dependent transformations.

arxiv.orgpaperAdded 16 Oct 2026, published 12 Oct 2026🛡️ Security

Presents Creed, an LLM-based code auditing approach that infers a developer's security-related beliefs for a function, scores how reasonable each is, and derives the conditions under which it is violated. It catches domain-specific semantic flaws that syntactic static analysis misses, such as a timing side-channel that undermines user-enumeration protection. The paper also introduces two measures for explanation-based vulnerability detection, recall and overhead. On a recent real-world benchmark, Creed beats existing explanation-based tools on the effectiveness-effort trade-off.

arxiv.orgpaperAdded 15 Oct 2026, published 11 Oct 2026🛡️ Security

Presents Creed, an LLM-based code auditing approach that, for a given function, infers the developer's security-related beliefs, scores how reasonable each belief is, and derives the conditions under which it is violated. This lets it find semantic flaws, such as timing side-channels in authentication, that syntactic static analysis tools miss, and it explains root cause, symptoms and exploitability. The paper also introduces two evaluation measures for explanation-based vulnerability detection: recall (how well a known vulnerability is correctly explained) and overhead (the effort of inspecting reports). On a recent real-world benchmark, Creed beats existing explanation-based tools on the effectiveness-effort trade-off.

arxiv.orgpaperAdded 15 Oct 2026, published 11 Oct 2026🧠 AI

Introduces Policy-Constrained Diffusion Agents (CoDA), which enforces task-dependent security policies inside the denoising process of diffusion language models. At each step, token distributions are projected onto a policy-compliant set, so constraints hold throughout generation instead of relying on detecting injected instructions. Evaluated across several dLLM families under static and adaptive prompt-injection attacks, it reports substantially lower attack success rates with competitive task utility and practical inference overhead.

arxiv.orgpaperAdded 15 Oct 2026, published 10 Oct 2026🛡️ Security

KedgeFlow specifies language-independent contracts so trusted controllers can authorize state-changing operations by enterprise AI agents and privileged gateways can enforce that authority independently of agent reasoning. It combines evidence-backed policy checks, signed permission for an exact action, a single-use shared allowance, target-side conditions and retained outcome records, with two conditional properties (at most one claim per allowance; every dispatch linked to a grant and claim) that hold even if the agent is compromised. A reference implementation exposes the path via MCP; evaluation on a three-member etcd cluster gave exactly one successful claim for each of 82 allowances across 6,560 concurrent attempts, and one committed effect in a 32-call gateway race. The authors report limits: permissive policy still authorized an undesirable effect, and distributed reservations, production access routes, crash recovery and real-target exactly-once effects remain untested.

arxiv.orgpaperAdded 15 Oct 2026, published 10 Oct 2026🧠 AI

KedgeFlow proposes language-independent contracts in which trusted controllers issue signed exact-action permissions and privileged gateways enforce them, using single-use allowance claims, target-side conditions and retained outcome records. It aims to stop altered approved actions, concurrent permission reuse and retries after lost responses, even with a compromised agent. A reference implementation exposes this path via MCP; evaluation on a three-member etcd cluster showed 6,560 concurrent claim attempts yielding exactly one success per each of 82 allowances. It also reports that permissive policy still authorized an undesirable effect, and lists untested areas such as distributed reservations and production access routes.

arxiv.orgpaperAdded 15 Oct 2026, published 10 Oct 2026🛡️ Security

The paper studies joint attention in multimodal diffusion transformers (MMDiTs), where the evolving image updates text representations. It finds that this image-to-text (I2T) feedback tends to preserve objects and attributes already in the image, errors included, and that suppressing it improves generation. It proposes Semantic Feedback Guidance (SFG), which guides away from an auxiliary branch that strengthens I2T feedback and weakens text-to-image conditioning, and iSFG, an in-place variant that needs no extra model evaluations. SFG runs for the first 20% of sampling steps (about 1.2x base NFE without CFG). Across three backbones, adding SFG to CFG on FLUX raises GenEval from 0.6661 to 0.7479, and iSFG on SD3.5M adds +1.33 HPSv2.1 over CFG at zero extra NFE. The control also improves VBench scores for text-to-video and image-to-video.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

Introduces EICS (Evaluation-Invariant Concept Stability), a metric for checking whether representations consistently encode a concept under controlled one-concept interventions. Also introduces CASP-VL, a benchmark of causally controlled spatial perturbations derived from Visual Genome scene graphs. Across several VLM architectures, spatial relations are represented much less faithfully than lexical information, even for models well above chance on compositional benchmarks. The relation is identifiable at the predicate's own tokens but is lost when the sentence is aggregated into a global representation (EOS token or mean pooling), a failure that output-based evaluation does not reveal.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

Proposes MASTIF (Multi-Agent System TestIng Framework) for evaluating Agentic Web stacks. It reports a baseline assessment of 3 LLMs, 3 protocols and 4 agent implementation frameworks on 300 Mind2Web benchmark tasks. Results suggest that choosing a task-specific stack can improve task completion rate, efficiency and social value stability, and can reduce verbosity and completion time. This gives developers reference metrics for choosing among fast-moving agent protocols and frameworks.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

BLiMPPI is a multilingual evaluation suite of over 12,000 negative polarity item (NPI) minimal pairs across 33 languages. A pipeline finds NPIs in Universal Dependencies treebanks, labels their licensing environments, and perturbs them into grammatical/ungrammatical pairs. Across 12 multilingual and 2 monolingual models plus n-gram baselines, multilingual models do worst on lower-resource languages and on pairs that n-grams cannot solve. The paper also analyses NPI typology, language family, licensing variability and NPI-licenser distance.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

ManeuverFlow is an end-to-end autonomous driving planner that generates trajectories for distinct driving intentions in one step. It selects maneuver modes, draws noise from a Gaussian mixture prior, and maps it to trajectories through a maneuver-routed mixture-of-experts MeanFlow field. Current features and world-model-predicted future features are aggregated along each candidate path as evidence for PDM-factor scoring and Frenet-residual refinement. With a TransFuser-style ResNet-34 backbone it reaches 90.6 PDMS on NAVSIM v1 and 90.1 EPDMS on NAVSIM v2 navtest. It also reports the highest listed PDMS* on WorldEngine's 3DGS closed-loop benchmark, 75.19 with logged traffic and 74.58 with IDM-controlled traffic.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

AgentTune is a source-available Python library that unifies building, training, evaluating, distilling and maintaining tool-using LLM agents. Its core is a two-tier EventLog representation that preserves agent trajectories and yields consistent projections for training, evaluation and auditing. On top of it sit a shared agent runtime, tool-calling reinforcement learning, agentic distillation, RAG, decision workflows, evaluation utilities and a designed path toward self-healing agents, demonstrated on tasks such as legal contract search.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

SmartSalesBench is a multi-turn conversational benchmark that tests LLM sales agents against psychologically grounded, resistant buyer personas over conversations of up to twenty turns. It has five agent profiles, twelve buyer personas and fifteen smartphone products across six price tiers, and runs four controlled experiments with 3 iterations (13,680 sessions), with RAG grounding both agent and buyer generations. It defines nine metrics in three tiers (primary CR, BRS, RIR; secondary SMS, PFS, ETI; exploratory ITS, ARI, SPI) and combines LLM-as-judge scoring with a human assessment (n=50, kappa 0.71, Pearson r 0.81 with BRS). It matters because it drops the cooperative-user assumption of existing task-oriented dialogue benchmarks.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

AgentValve is a protocol for authorizing payments by AI agents in permissionless markets where adversaries can cheaply create Sybil identities and fake feedback to inflate provider reputation. It models reputation as finite capacity in a directed trust graph, and a smart contract atomically reserves capacity along trust paths for each admitted payment. This bounds honest agents' exposure to the capacity they extend into the adversarial region, and payment-backed capacity creation with bounded route length limits depletion attacks. A witness-commit mechanism makes on-chain route verification linear work with constant persistent state per payment. The prototype was evaluated on six months of Ethereum data and synthetic workloads with up to 10,000 Sybil identities. Exposure and depletion stayed bounded, admission cost fell by up to 40.8% against baselines, and common payments cost about $0.04–$0.21.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🛡️ Security

AgentValve is a protocol for Sybil-resilient payment authorization when AI agents pick and pay service providers without human review in permissionless markets. It models reputation as finite capacity in a directed trust graph, and a smart contract atomically reserves capacity along trust paths. This bounds honest agents' payment exposure to the capacity they extend into the adversarial region, so fake identities and fabricated feedback inside that region cannot raise the bound. Payment-backed capacity creation and bounded route length limit depletion by malicious senders. A witness-commit mechanism splits off-chain route discovery from on-chain verification, cutting verification to linear work with constant persistent state per payment. A prototype evaluated on six months of Ethereum data and synthetic workloads, with up to 10,000 Sybil identities, kept exposure and depletion bounded. Admission cost fell by up to 40.8% versus baselines, at roughly $0.04-$0.21 per common payment.

arxiv.orgpaperAdded 13 Oct 2026, published 9 Oct 2026🧠 AI

The paper shows that reward hacking learned by one coding agent during RL against automated evaluators can spread to other agents in a multi-agent system. Across four open-weight backbones, coding RL elicits and amplifies hacking when the system prompt describes the mechanism explicitly and when mid-training exposes the model to facts about the coding environment. Post-RL source agents transmit the hack through code, shared memory and natural-language plans, and even small open-weight agents induce hacking in stronger closed-weight targets. On hard tasks the hack can intensify along a transmission chain after the source leaves, as the code accumulates more persuasive justifications. A theoretical analysis gives conditions under which hacking amplifies, persists or decays. Hacking also biases task routing toward hacking agents and survives candidate aggregation. Code review by the coder or by a separate verifier reduces the hacks that get delivered.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

DecepEval is a benchmark of 1,532 instances across 3 task families and 28 professional scenarios designed to measure deception in LLM agents, pairing neutral and induced versions of each task to isolate condition-dependent changes in deception rates. It introduces the LLM Deception Diamond framework (pressure, incentive, opportunity, conflict), adapted from classical fraud theory, and finds that across nine frontier LLMs these inducements raise deception rates even in models with low baseline deception, with long-horizon tasks showing deception rates averaging 87% under induced conditions.

huggingface.copaperAdded 8 Oct 2026🧠 AI

OuroWorld is a mask-free framework that turns any static 3D Gaussian Splatting scene into a seamlessly looping 3D cinemagraph. A vision-language model infers plausible dynamics and guides a video model to synthesize a reference video, which is lifted and completed into multi-view videos. Inconsistency-Robust Periodic 4DGS then fits these imperfect videos: a Fourier-series deformation field guarantees looping by construction, and a Grounded Drift Field anchored at the reference view absorbs cross-view inconsistency. It handles general deformation, object motion and illumination change, not only fluid-like motion as prior Eulerian methods do. The paper also proposes a ground-truth-free evaluation covering vividness, naturalness, loop seam coherence and scene quality. On 39 reconstructed and generated scenes it beats all baselines and wins 70.8%-99.0% of user-study comparisons.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

OmniCapBench is a benchmark of 786 densely annotated videos that evaluates audio-visual captioning in multimodal LLMs. Instead of scoring free-form captions, it asks models for sets of atomic, verifiable units across three tracks (entity references, visual shots, audio events). These are scored with deterministic constraint checks plus localized LLM-based semantic comparisons, which gives coverage, localization and stable scoring together. Tests on frontier MLLMs show strong local perception but weak long-horizon audio-visual reasoning, especially identity drift and cross-modal misalignment.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

WOVEN is a training source and benchmark for visual transition reasoning in multimodal LLMs. It has 36,076 examples spanning 20 scene types, 5 action types and 8 reasoning types, built from rollouts of video-pretrained generative models. Evaluating 38 frontier MLLMs (including GPT-5.4 and Qwen3-VL-235B-A22B) shows systematic deficits that persist with scale. Training on WOVEN subsets of about 2,000 items improves 22 of 26 external benchmarks by up to 27.3 points, and WOVEN data can replace 30-50% of a task's own training data. The authors derive a recipe: select supervision by the reasoning operation it teaches rather than by scene or action, and prefer larger visual-state changes.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

ORCAGen uses RAG over a curated knowledge base of malware procedures and active-defense strategies, plus structured prompting, to have LLMs generate proof-of-concept malware and matching deception orchestration code offline. Strategies are validated against the PoC malware before being added to a runtime playbook that enforces only the verified, deterministic logic. It is evaluated on GPT-4o, GPT-5.5, Gemini 3.5 Flash, Qwen3-Coder and Claude Sonnet 4.5 over synthesized scenarios and 150 real keylogger, infostealer and ransomware samples, measuring execution success, hallucination rate, refinement effort, deception effectiveness and runtime overhead.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🛡️ Security

The paper proposes evaluating LLM agents on epistemic humility, meaning their willingness to recognize, act on and communicate uncertainty. It operationalizes this as three trajectory-level dimensions (Identify, Solve, Escalate, or ISE). It tests controlled and naturally occurring knowledge conflicts, each with matched no-conflict controls, on four agents. Higher accuracy does not imply more humility: some high-accuracy agents detect conflicts but do not communicate uncertainty in wrong final answers, and agents often detect conflicts early but lose track of them later. Model-level interventions can improve humility but often cost accuracy, which suggests humility depends on the interaction of backbone model, agent harness and environment.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

ReSI is a recursive safety improvement framework. Each round runs diverse red-teaming methods against the current target model, develops training recipes for the vulnerabilities found, and promotes the update with the largest safety gain among those passing a Pareto gate on capability retention. Across four dense and mixture-of-experts models it cuts mean X-Teaming multi-turn jailbreak attack success from 86.01% to 31.45%, versus 56.69% for the best frontier model evaluated (GPT-5.6-Luna). It also beats alignment baselines on nearly all safety evaluations while largely preserving general capability. It frames the goals as resistance to known threats and resilience to unforeseen ones, following R²AI.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🛡️ Security

Introduces AgenticBBO-Bench, a benchmark for LLM-agent black-box optimization across synthetic functions, hyperparameter optimization, database tuning, chip design and molecular design under a unified finite-budget protocol. Agentic BBO beats direct LLM methods in all five domains and the best numerical optimizers in four. Ablations show extra numerical tools don't consistently help, task semantics help more than specific priors, and numerical optimizers can absorb gains from agent-found trajectories. A five-task frontier challenge evaluates seven LLMs under the Codex agent harness, with GPT-6 Astra and DeepSeek-V4.1-Flash on the performance-cost Pareto frontier.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

Paper measuring prefix-based training-data extraction in instruction-tuned open-weight LLMs under an empty prompt versus retrieved contexts of varying relevance (RAG), using paired item-level suffix scoring across three models. It finds extractable memorization has a context-robust core and a context-sensitive boundary: context suppresses some exposures but also enables new ones missed by prefix-only evaluation. This qualifies the claim that RAG reduces memorization risk and matters for threat-modeling deployed LLM systems.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🛡️ Security

The paper measures whether retrieved context, as in RAG, reduces LLM memorization. It does this with paired item-level probabilistic suffix extraction: each prefix-suffix pair is scored under an empty prompt and under retrieved contexts of varying relevance, across three open-weight instruction-tuned models. Extractable memorization splits into a context-robust core and a context-sensitive boundary. Context suppresses some marginal exposures but also enables new ones that prefix-only evaluation misses, which qualifies the claim that RAG lowers memorization risk.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

Evaluates Jev, a "System One" decision model that returns a choice among given options instead of generating text, on 13 multiple-choice benchmarks covering knowledge, reasoning and multilingual understanding. It is compared with 19 LLMs in frontier, representative and small tiers. Jev is competitive with frontier LLMs on knowledge and commonsense tasks, and best on MMLU-Redux and ARC-Challenge. It beats most representative LLMs and all small ones outside math, but is 17.7 points below the frontier median on MathQA and scores lower than all 19 LLMs there. This suggests specialized decision models can match general LLMs on knowledge-driven decisions but not on multi-step calculation.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

Proposes "study contracts" that bind an AI-generated study's declared experimental choices, run obligations and claim scope to recorded execution evidence, defining contract-relative verification as distinct from scientific truth. In a diagnostic with eight self-authored clean/mutated pairs, a deterministic fault-specific checker caught all eight mutations, while eighteen LLM judge aliases given metadata-filtered packages flagged 104 of 144 mutated cases (32 abstentions, 8 terminal failures). The authors stress the setting is deliberately information-asymmetric, not a verifier-quality comparison or reward-hacking result, and call for full-information comparisons, legitimate-adaptation controls and closed-loop agent evaluations.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

TRACE (Train-Rollout Quantization Alignment via Compact GuidancE) is an FP4 quantization framework for reinforcement learning post-training of Mixture-of-Experts language models. It uses rollout-guided quantization-aware training to align training-side FP4 rounding with rollout-side quantization outcomes, reducing the train-rollout discrepancy that limits prior low-precision RL methods, and adds a caching scheme that selectively retains mantissa/scale information from deeper layers to cut communication overhead. Evaluated on four large MoE models across reasoning, coding and long-horizon RL tasks, it matches BF16-rollout RL performance while achieving up to 5.4x rollout speedup with joint FP4 weight/activation and FP4 KV-cache quantization.

arxiv.orgpaperAdded 7 Oct 2026🧠 AI

This paper studies where tool-using LLM agents break down in the chain from gathering evidence to taking consequential actions, finding that strong static judgment of whether to act often coexists with much weaker interactive execution, and that failures arise from incomplete investigation, acting before evidence is established, or unresolved prerequisites in multi-action workflows. The authors introduce SafeActBench, a 656-case benchmark across six operational domains and five protocols, paired with a provenance-bound Evidence Ledger and a deterministic trajectory evaluator tested across ten model-harness configurations.

arxiv.orgpaperAdded 7 Oct 2026🧠 AI

BRANCH is a bypass methodology for guardrail systems built from multiple scanners (prompt injection/jailbreak detectors). It uses a branching tree search that applies adversarial perturbations to individual scanners and selects techniques based on overall improvement across all scanners, decoupling bypass evaluation from attack signal optimization. It reports 100% attack success across 6 guardrail systems in 120 scenarios with 72% fewer queries and 4.5x lower wallclock time than established techniques, and transfers to 29 unseen guardrails, including 8 commercial black-box ones.

arxiv.orgpaperAdded 9 Oct 2026, published 7 Oct 2026🛡️ Security

LOCAA is an LLM-based agentic system that automatically tunes lossy scientific data compressors by performing compression-in-the-loop search, using tool-integrated execution, compressor-aware guidance, and persistent memory instead of exhaustive parameter search. Evaluated across three scientific applications and five compressors on fixed-ratio compression and multi-constraint tuning tasks, it cuts evaluation trials by 1.98x–5.03x versus binary search and FRaZ, and persistent memory further reduces trials by 27.9% across repeated timesteps of the same field.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

This paper proposes three evaluation screens — Decisive-Action BPB, Patch MCQ, and prefix-conditioned pass@K — that predict how well a base model will perform as a coding agent after post-training, without requiring the base checkpoint to run a full agentic harness. By replaying successful post-trained agent trajectories and identifying the "decisive step" that flips a repository from failing to passing tests, the authors measure a base model's latent support for that action. Tested across ten pairs of public base and post-trained models, all three screens closely track post-trained SWE-bench Verified pass@1 scores, offering a way to cheaply vet base checkpoints before committing to expensive agentic post-training.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

This paper introduces a detector for adversarial images targeting frozen vision-language models (VLMs) by building a 'response profile' from how an image interacts with a bank of general semantic prompts — using category-level statistics, inter-prompt relationships, deviation from clean reference distributions, and stability under weak image transforms, then classifying that compact profile with a lightweight model. Evaluated across multiple public datasets, several CLIP-style visual backbones, and gradient-based, optimization-based, automated, and spatial attacks, the method outperforms embedding-geometry baselines, generalizes partially to unseen attacks, and is tested against detector-aware adaptive attacks.

arxiv.orgpaperAdded 9 Oct 2026, published 7 Oct 2026🛡️ Security

Evaluates delivering security requirements from a governed security-by-design knowledge base (SbD-ToE) to a code-generating LLM through an MCP server at generation time. On DualGauge (59 Python tasks) the share passing all security tests rose from 44.1% to 78.0%; on BaxBench (28 backend scenarios) the share of functionally correct solutions with no successful exploit rose from 65% to 86%, matching the authors' Oracle Security Reminder (85%). The joint secure-and-functional metric did not improve significantly, since stricter code failed functional tests that fix unstated values. The study used one model and one generation per task, and it proposes measuring conformance with applicable requirements rather than only detecting known weaknesses.

arxiv.orgpaperAdded 9 Oct 2026, published 7 Oct 2026🛡️ Security

The preprint tests whether security requirements from a governed security-by-design knowledge base (SbD-ToE), delivered to a code generator through a Model Context Protocol server at generation time, make AI-written code more secure. On DualGauge (59 Python tasks, LLM-judge scoring) the share of tasks passing all security tests rose from 44.1% to 78.0%. On BaxBench (28 backend scenarios with real exploits in containers) the share of functionally correct solutions with no successful exploit rose from 65% to 86%, comparable to the authors' Oracle Security Reminder (85%). The pre-specified joint secure-and-functional metric did not improve significantly, and the stricter code failed functional tests that fix values the task never states. The study used one model and one generation per task. The authors argue that current benchmarks only detect the weaknesses their evaluators were built for, and propose measuring conformance with the requirements that apply.

arxiv.orgpaperAdded 9 Oct 2026, published 7 Oct 2026🧠 AI

The paper formalizes speculative decoding as a Markov reward process and derives the Expected Decoding Rounds (EDR) objective, a hyperparameter-free training loss equal to the expected number of decoding rounds, addressing cross-round coupling that prior block-local surrogate objectives ignore. It derives an exact temporal-difference gradient for unbiased stochastic optimization and an offline evaluator for comparing drafters on shared rollouts without running speculative decoding. Finetuning the DSpark and DFly drafters with EDR improves mean accepted length and beats prior training objectives across nine benchmarks in math, code, and chat.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

TaoD2C-Bench is a benchmark for evaluating multimodal large language models (MLLMs) on industrial design-to-code (D2C) tasks — converting UI designs into code that satisfies implementation constraints, not just visual fidelity. It includes the TaoD2C dataset of 2,861 production designs from 17 commercial platforms with 97,652 expert annotations across Component, Group, Alignment, and Position categories, and defines three tasks: end-to-end UI code generation, requirement inference, and requirement realization. Evaluating eight MLLMs, the authors find large gaps between visual reconstruction ability and the ability to generate code meeting domain-specific implementation requirements.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

SLDR is a post-fine-tuning defense that protects aligned LLMs from malicious fine-tuning-as-a-service attacks, which can quietly erode refusal behavior while keeping task performance intact. It trains a LoRA recovery adapter only on the layers with the most extreme (signed) safety sensitivity scores, then uses representation-based dynamic routing to activate that adapter solely for malicious queries at inference time. Evaluated across four model architectures, five downstream tasks, and four harmful-content benchmarks, it drives the average harmful score on Llama3.1/SST2 from 11.54 down to 0.08 while preserving downstream accuracy, and stays near zero even at poisoning ratios up to 0.9.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

This arXiv paper examines why current LLM-based automated vulnerability-patching benchmarks overstate model performance, identifying pitfalls at the agent level (prompting/tooling inflating success without improving patch quality), framework level (permission errors, infra bugs, timeout handling silently skewing scores), and dataset level (single PoC tests failing to verify root-cause fixes). The authors curate 112 historical bugs across 84 open-source C/C++, Go, and Rust projects, each annotated with PoC tests, regression tests, and developer-intent tests, and show that while PoC pass rates look high, developer-test pass rates stay low and barely improve with newer models. They provide guidelines for more rigorous, reproducible evaluation of LLM patching systems.

arxiv.orgpaperAdded 9 Oct 2026, published 7 Oct 2026🛡️ Security

This arXiv paper proposes foundations for proactive LLM agents organized around three joint principles (3T: Task Capability, Temporal Allocation, Trust) and a five-dimensional design space (task scope, anticipation horizon, activation trigger, processing timing, intervention depth). It introduces PROACTIVITY-GYM, a simulation-based testbed with multi-day scenarios, stateful environments, and persona-conditioned simulated users, and reports evaluations across 23 model-harness configurations plus a 30-participant human study showing sharp trust declines after intervention misalignment and a preference for imperfect sleep-time assistance over interruptive help.

arxiv.orgpaperAdded 6 Oct 2026🧠 AI

World Models' Last Exam in Physics introduces a measurement-based benchmark of 40 controlled tasks across mechanics, optics, fluids, thermal/phase-change phenomena, electromagnetism, and surface tension, designed to test whether video world models generate physically consistent outputs. Each task pairs an initial image and prompt with predefined physical criteria, and a custom evaluator combines observability screening with task-specific quantitative measurements rather than relying on reference videos or model-based judgments. Testing eight video generation models across 1,280 generated videos found persistent physical inconsistencies, with the best model scoring only 57.76/100, and the evaluator showed higher agreement with human judgments than a vision-language model baseline.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

This arXiv paper introduces a post-training semantic lifting method for 3D Gaussian Splatting that assigns a single target class to Gaussians by accumulating target/non-target evidence across all views, weighted by per-view visibility, then filters with a dual-threshold (β and γβ) scheme that selects high-confidence seeds and extends them via connected components. The authors evaluate by transferring labels from Gaussians to mesh vertices and decompose error into three sources — the 2D detector, the lifting step, and the representation transfer — tuning thresholds on seven Replica scenes and testing on ten held-out ScanNet++ scenes, reaching mean mIoU of 0.93/0.65 (dataset masks/YOLO masks) on validation and 0.80/0.54 on test, a 0.24 mIoU improvement over prior per-view thresholding. The error analysis finds the 2D detector is the dominant remaining error source, informing where future gains in 3D semantic reconstruction should focus.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

ScienceClaw formalizes a fixed-parameter program self-evolution framework for LLM agents doing scientific tasks, where verified task executions are turned into persistent Skill and Operator updates to the agent's program rather than being discarded. ScienceClaw-Eval, the accompanying benchmark, spans 23 disciplines across natural and social sciences and measures scientific correctness, evolutionary gain, retention, cross-dataset transfer, and evolution cost via sequential task streams and independent reset evaluation. The framework only keeps an update if source-task replay reproduces the repair and the change improves performance on independent scientific tasks, addressing the gap that verified agent executions rarely become lasting program-level improvements.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

This paper presents a systematic empirical study of small language models (decoder-only, encoder-decoder, and mixture-of-experts architectures) on abstract grid-transformation reasoning, using the ARC-TGI benchmark, which supports controllable task families, resampling, spatial shifts, and cross-benchmark transfer to distinguish genuine rule acquisition from distribution-specific overfitting. Across more than 1,000 supervised fine-tuning runs, the authors find that in-distribution accuracy is achievable but highly sensitive to optimization and task formulation, performance collapses sharply out-of-distribution (even under simple grid-scale changes with the same underlying rule), training-set scale yields uneven gains, in-context example benefit is model-family dependent, and executable-rule induction surfaces correct solutions that direct grid generation misses. Layer-wise attention diagnostics reveal behavioral differences in concentration and context-dependence but stop short of establishing causal mechanisms, underscoring that abstract-reasoning scores are conditional on model, adaptation regime, evaluation distribution, and output format.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

EC-RAG (Event Chain Retrieval-Augmented Generation) is a training-free framework for long-video question answering that organizes video content into an explicit, temporally ordered "event chain" instead of retrieving isolated frames or snippets. It partitions video into semantically coherent segments, fuses multi-modal signals (speech, text, visual cues) at the event level, and retrieves relevant events from the chain to answer queries, working as a plug-and-play addition to existing large video-language models (LVLMs) without extra training. Evaluated on Video-MME, MLVU, and LongVideoBench, it outperforms frame-level retrieval baselines, showing the value of modeling temporal/event structure for long-video understanding.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

This paper introduces Semantic Behavioral Watermarking (SBW), a method for embedding an owner identifier into an LLM agent's high-level tool/action choices rather than output tokens, so provenance survives tool renaming and observation paraphrasing. It watermarks over semantic action clusters with history conditioning and replaces the public-cluster bin with keyed collision-resistant binning that is provably unpredictable under the random-oracle model, closing a forgery gap left by prior agent watermarks (e.g., AgentMark, whose bit-recovery drops to 16.8% under paraphrasing). Evaluated across five agent models (3B-14B, four vendors) on ToolBench and ALFWorld, cluster-level watermarking achieves far higher detection under rewriting (0.49-0.66 vs 0.05-0.17 on ToolBench; 0.92-0.97 vs 0.00-0.01 on ALFWorld) and reduces adaptive forgery success from 100% to the false-positive floor, while noting chained-replay attacks remain an open vulnerability (0.76-0.98 detection failure).

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

RAG-PIBench is a benchmark of 4,876 contextual examples for evaluating prompt-injection detectors embedded in retrieved content within Retrieval-Augmented Generation (RAG) systems. The authors use a leakage-aware construction pipeline with frozen train/validation/protected-test splits and compare keyword-based, semantic-reference, TF-IDF, and transformer-based detectors, finding DistilBERT achieves the best protected-test performance (F1 = 0.896, PR-AUC = 0.968) while TF-IDF with SVM or logistic regression remains competitive. It highlights the risk that naive benchmark construction can leak information between splits, inflating reported detector performance for RAG prompt-injection defenses.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🛡️ Security

This IACR ePrint paper proposes a security-aware benchmarking framework for deploying memory-heavy PQC (ML-KEM, ML-DSA) on the dual-core ESP32-S3 (Xtensa LX7) running FreeRTOS, aimed at smart-grid substation devices bound by the IEC 61850 GOOSE tripping window (≤3ms). It details an asymmetric task-isolation design pinning crypto to Core 1, a heap-auditing methodology for FreeRTOS's heap_4.c allocator tracking key-state allocations across DRAM, and an analytical latency model of CPU stall overhead caused when side-channel masking forces secret polynomial shares to spill to external Octal-SPI PSRAM during non-sequential Cooley-Tukey NTT strides. The work gives embedded engineers a methodology to evaluate PQC side-channel/latency trade-offs before migrating real-time grid infrastructure.

eprint.iacr.orgpaperAdded 8 Oct 2026, published 6 Oct 2026🛡️ Security

This IACR ePrint paper examines the post-quantum security of Bitcoin bridges that rely on off-chain garbled-circuit evaluation of succinct proofs to validate withdrawals, where input labels encoding the proof must be committed on-chain. The authors show current pairing-based proving systems are vulnerable to quantum attacks, and that swapping to post-quantum proving systems shrinks circuit size but drastically inflates proof size, forcing proofs to span multiple Bitcoin blocks and raising on-chain cost. They outline research directions to improve proof efficiency, circuit representation, and label encoding for quantum-resistant bridge designs.

eprint.iacr.orgpaperAdded 8 Oct 2026, published 6 Oct 2026🛡️ Security

This paper argues that the "Translation-Isomorphism Assumption" underlying multilingual LLM benchmarks is not just violated in practice but ill-posed for a class of concepts (pragmatic markers, honorifics, diachronically stratified terms). The authors formalize "α-unalignability" using a usage-cloud framework of contextualized embeddings, showing via FLORES-200 translation failures, LOBSTER reasoning scores, and a nine-model case study on the Yami language that models encode cross-lingual regularities (e.g. Yami clustering with other Austronesian/low-resource languages) that are not actionable via neuron-level interventions (a "Representation-Intervention Gap"). They propose a multidimensional diagnostic profile (Cycle-Consistency, Pragmatic-Load Disagreement, Manifold-Curvature Mismatch, RIG) to replace single-scalar cross-cultural evaluation.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

An IACR ePrint paper presenting K-LMS, an experimental variant of the NIST-standardized LMS hash-based post-quantum signature scheme in which the underlying hash is swapped for Korean primitives: the LSH-256 hash function (reference and vectorized) and a Tandem-DM double-block-length construction built from the CHAM, LEA, and ARIA block ciphers. The authors implement a non-invasive backend-swapping bridge over the reference LMS code and benchmark keygen/sign/verify latency, key/signature sizes, memory, and parameter sweeps, finding that hash substitution doesn't change key/signature sizes but that two measurement pitfalls (hash dispatch overhead, and software vs. hardware-accelerated SHA-256 baselines) can shift performance comparisons by an order of magnitude or 6x respectively. They report LSH-256 K-LMS signing on par with SHA-256 LMS in software, and 3.3-3.9x faster than a prior K-XMSS implementation because LMS needs fewer hash calls per Winternitz step than XMSS.

eprint.iacr.orgpaperAdded 8 Oct 2026, published 6 Oct 2026🛡️ Security

test

arxiv.orgpaperAdded 7 Oct 2026, published 5 Oct 2026🧠 AI

This paper runs a paired, reproducibility-focused evaluation of two System-1 single-forward-pass decision models, open-weight Laya and hosted Jev, across 11 typed decision points that agent harnesses make constantly: tool selection, RAG relevance gating, prompt-injection detection, and zero-shot model routing, using 7,283 base cases plus 6,640 robustness variants from 18 public sources. Jev beats Laya on 9 of 11 decision points by 10.8-46.0 percentage points, both models fail at zero-shot model routing, and Laya flips 30 percent of answers under option reordering and degrades badly with many or similar candidates. The authors also self-audit their own pipeline, finding and correcting three analysis errors and a design confound, including a reported 23.9 percent cost saving that was actually 4.3 percent, and gate accuracy conflated with end-to-end quality.

arxiv.orgpaperAdded 5 Oct 2026🧠 AI

OVAL (Output-Aware Local page bases) is a training-free method for encoding KV cache pages in page-sparse attention, designed so that retrieval directly accounts for how approximation errors affect the value-weighted attention output rather than just estimating attention scores. It derives page representations from the joint structure of keys and values while keeping the same storage size and decode-time scoring cost as a key-only spectral baseline, and is evaluated on long reasoning, long context understanding, and long generation benchmarks where it outperforms the key-only spectral baseline and matches or beats recent KV cache compression/retrieval methods with modest decoding overhead.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

FREA is a benchmark of 751 chemical reactions labeled by expert chemists for feasibility verification, drawn from retrosynthesis model proposals, zero-yield experimental records, LLM-generated edits, and five negative-candidate generation methods. The authors show no single verifier type (LLMs prompted with the feasibility criterion, dedicated forward models, or retrosynthesis scorers) dominates across all sources, and that adding generated negative examples to training improves mean AUROC but fails to transfer to retrosynthesis proposal screening. They also release a corpus of over 14 million recorded and generated reactions to support further training and evaluation of feasibility verifiers.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

Lens3D is a method for improving fine-grained object understanding in 3D large language models by using 3D localization to select informative 2D views that are fed to an external vision-language model (the LensUnd pipeline), then distilling that knowledge back into native 3D LLMs via detailed caption supervision (LensDistill), avoiding the need for external VLM calls at inference time. The authors also release LensBench, a held-out evaluation set of 2,068 objects with three reference descriptions each, and show gains in fine-grained object captioning on Video-3D LLM and 3DRS while preserving grounding and scene-level QA performance.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

BrainTRACE is a report-grounded benchmark for evaluating whether vision-language models can trace the evidence structure required for longitudinal brain MRI interpretation. It contains 7,273 scored VQA instances derived from 1,778 longitudinal patients, 7,299 MRI studies, and ~29k co-registered 3D MRI sequence volumes, organized across five levels of clinical reasoning and evidence demands like longitudinal comparison, multi-sequence integration, and volumetric spatial grounding. Evaluation of 20 VLM configurations finds that current models identify isolated visual cues but rarely compose them into grounded longitudinal clinical interpretations.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

ANT (Agent Network Traffic) is a dataset and benchmark for auditing LLM agent behavior purely from observed network traffic, without inspecting private user content. It contains 3,114 execution episodes across 20 tasks and five scenarios, with 276,417 bidirectional flows and 40,049 behavior-primitive segments grouped into 47 macro categories, and evaluates 13 traffic-analysis baselines on risk identification, scenario recognition, and behavior-primitive classification. Results show existing methods recover useful but uneven signals, struggling especially when malicious agent workflows resemble benign ones or scenarios share similar traffic patterns.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

SpatialChain is a benchmark of 28,350 training and 899 test examples pairing spatially-oriented GQA questions with scene-graph-grounded reasoning chains, designed to audit whether VLMs' correct spatial answers reflect faithful reasoning or linguistic shortcuts. Using a two-axis evaluation (chain-overlap metrics plus a scene-graph-aware LLM judge), the authors find that four of nine tested models hit ≥79% accuracy while showing shortcut rates above 39%, that chain quality predicts correctness for most models except Claude Sonnet 4.6 and InternVL3.5-8B (which fail differently, via terse vs. verbose-decorative reasoning), and that SFT on SpatialChain improves Qwen3-VL-8B accuracy by +6.2pp while cutting its shortcut rate to 22%. The faithfulness judge is validated against human annotations and a second LLM judge, with data, generation scripts and evaluation code released publicly.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

This IACR ePrint paper benchmarks 20 open-source Gröbner basis implementations against the proprietary Magma system on polynomial systems derived from small-scale AES over GF(2), using Easy/Medium/Hard difficulty tiers with ten instances each. In the Hard setting, msolve and Singular's slimgb completed all runs successfully with mean runtimes of 164s and 269s and peak memory of 3693/4343 MiB respectively, versus Magma's 319s and 16853 MiB, with msolve exploiting ~13.4 CPU cores versus ~1 core for the others. The results quantify when open-source algebraic solvers are practical alternatives to Magma for algebraic cryptanalysis of block ciphers.

eprint.iacr.orgpaperAdded 8 Oct 2026, published 5 Oct 2026🛡️ Security

SatBleed is an arXiv paper presenting the first in-depth security evaluation of commercial off-the-shelf (COTS) Communication Subsystem (COM) modules used in small satellites. The authors build a threat taxonomy for COM-targeting attacks, analyze representative COM hardware from multiple vendors, and uncover severe firmware, protocol, and architectural vulnerabilities. By correlating findings with open-source telemetry data, they identify at least 28 vulnerable missions currently in orbit that are susceptible to hostile takeover via their command-and-control link.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🛡️ Security

This paper proposes AI-decision checkpoints, explicit moments in business process development where process developers must evaluate whether to adopt, constrain, or reject AI-candidate sub-processes built with LLMs, RAG, or AI agents, based on expected effects on time, cost, quality, flexibility, and legal or organisational constraints. The checkpoints are embedded in a six-module BPM teaching lifecycle covering process modeling, simulation, workflow execution with AI agents, and process mining, piloted via a fictitious customer-onboarding teaching case with formative observations drawn from instructor logs and student artifacts. It addresses a curriculum gap where AI is still treated as a bolt-on rather than a first-class design element in BPM education.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

IACR ePrint paper constructing and implementing the first one-round threshold signing protocol for the hash-based post-quantum signature schemes XMSS and SPHINCS+, producing signatures that verify unmodified against RFC 8391 and FIPS 205. The approach shares the secret key among t-of-n signing parties and evaluates the signing algorithm homomorphically using threshold FHE over discrete CKKS, achieving 3.4ms amortized per hash and 0.69s online threshold XMSS signing.

eprint.iacr.orgpaperAdded 8 Oct 2026, published 5 Oct 2026🛡️ Security

This paper introduces a paired-replay testbed that samples a single LLM agent request and replays the identical malicious action, resource, and arguments under four authorization conditions: broad bearer tokens, scoped JWTs, sender-constrained tokens, and Open Policy Agent policies. Across 128 scenarios in four tool domains and five local model configurations, broad-bearer tokens let roughly 9 to 38 percent of attacked, post-exposure decisions execute harmful actions, while all three scoped conditions recorded zero harmful executions; a supplementary AgentDojo extension found 11 of 24 injected attacks succeeded under broad policy versus 0 of 24 under scoping. The authors frame this as evidence that scoping contains cross-action and cross-resource consequences of prompt injection, not that it prevents the injection itself.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🛡️ Security

This paper proposes a fail-closed "bounded provisional visibility" protocol for continuously ingested RAG vector stores, which admits new content under a verification deadline and hides items whose vetting hasn't committed in time, bounding how long unvetted (potentially poisoned) content can be retrieved. Evaluated on five document workloads using an exact-match backend and Milvus with HNSW indexing, the deadline-based admission cut median poisoned retrievals from 34 to 7 under verifier backlog versus asynchronous admission with no deadline, while verify-before-visible avoided exposure entirely but delayed clean content visibility to 6.3s versus 3ms for provisional admission. The work quantifies the inherent trade-off between freshness, availability, and poisoning exposure in continuously updated retrieval systems, and shows results remain conditional on the quality of the underlying poison detector.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🛡️ Security

Agentic-ZTA is a multi-agent architecture that operationalizes the NIST SP 800-207 Zero Trust Architecture control loop using coordinated AI agents. A Policy Enforcement Point intercepts access requests, enriches them with context, and routes them to a policy-engine agent that invokes domain-specialized core and supporting agents; policies are retrieved via RAG and embedded into agent prompts at inference time, with trust scores aggregated by a trust algorithm to produce the final access decision. The authors implement and evaluate the framework on a testbed, reporting 95.0% accuracy, 93.9% precision, and 96.3% recall on representative access-control scenarios.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🛡️ Security

An IACR ePrint paper presenting an architecture-aware implementation of the X-Wing hybrid KEM (ML-KEM-768 plus X25519) for the ARM Cortex-M85, using its M-profile Vector Extension NTT and a shared-inversion trick during encapsulation to cut cycle counts by 12 to 19 percent versus a baseline, at a cost of about 35KB extra flash. The authors found that their optimized variant introduced secret-sign-dependent operand-address selection, a side-channel weakness, and produced a corrected version whose fix is only partial since some key-varying timing signals persist and remain unexplained, so no constant-time guarantee is claimed.

eprint.iacr.orgpaperAdded 8 Oct 2026, published 5 Oct 2026🛡️ Security

This IACR eprint paper studies output policies for CKKS homomorphic-encryption-based anomaly detection, evaluating whether limiting released scores and decisions can hide a private detection threshold without breaking existing decisions. Across 45 plaintext and 45 CKKS experiments on three intrusion-detection datasets and autoencoder scoring models, the authors show thresholds remain estimable, quantization preserves about 99.86 percent of decisions, and a surrogate model trained on only 128 decision-only responses achieves 0.817 mean balanced accuracy at predicting decisions on new inputs, a decision-prediction model-extraction-style attack whose success depends heavily on data-splitting strategy. It argues numerical threshold leakage and decision-prediction leakage are distinct protection goals that output policies for encrypted intrusion detection systems must address separately.

eprint.iacr.orgpaperAdded 8 Oct 2026, published 5 Oct 2026🛡️ Security

An IACR eprint paper evaluating all 13-cubed ways to assign 13 post-quantum/traditional signature parameter sets across the Root, Intermediate and Leaf roles of a TLS 1.3 certificate chain, holding the algorithm set fixed and varying only placement. Across 286 three-algorithm groups, median max/min ratios reach 2.085x for TLS handshake latency, 4.310x for certificate body size and 4.551x for path-validation cost purely from role order, with Leaf placement mattering far more than Root/Intermediate swaps. It also shows that looking only at Certificate-message size can misrank candidates such as ML-DSA-65 versus SLH-DSA-SHAKE-192s once the Leaf signature in CertificateVerify and full authentication-path cost are included, arguing PQC certificate-chain design must account for role-specific cost under real network/load conditions.

eprint.iacr.orgpaperAdded 8 Oct 2026, published 5 Oct 2026🛡️ Security

This paper evaluates whether Korean post-quantum crypto candidates (HAETAE, AIMer, SMAUG-T, NTRU+) and NIST PQC standards (ML-KEM, ML-DSA, SLH-DSA) can replace classical ECDSA/ECDH in the authentication and key-establishment path of ROS2/DDS-Security for UAV/UGV command channels. The authors build a feasibility envelope mapping each algorithm's key/signature size and handshake latency against real command-link constraints (message rate, MTU, latency deadline, bandwidth), then validate it with authentication-handshake measurements over emulated constrained links on a two-node testbed, producing a classical→NIST→KpqC comparison and per-channel-budget application guidance.

eprint.iacr.orgpaperAdded 8 Oct 2026, published 5 Oct 2026🛡️ Security

This paper isolates the coding-agent harness as an experimental variable by running the same model weights and tasks through two configurations of a custom harness called Yuj: a control that feeds the full chronological transcript until it hits the context window, and a treatment that keeps a complete in-memory record but shows the model a shrinking view (aging tool-result truncation past 50% window fill) plus a stall/loop detector with mechanical interventions. On SWE-bench Verified, SWE-bench Pro, and FeatureBench under tight context, the treatment raises mean per-task fail-to-pass fraction (F2PF) and complete-resolution counts (e.g. 28->49% F2PF, 43->72 resolved tasks on a 169-task Verified cohort at a 20,480-token window), with gains holding across additional models without retuning. The authors argue coding-agent evaluations should treat model+harness as the tested solver rather than attributing results to the model alone.

arxiv.orgpaperAdded 4 Oct 2026🧠 AI

RubricArmor is an adversarial framework for generating evaluation rubrics used in rubric-based reinforcement learning of LLMs, designed to preempt reward hacking before RL training begins. It alternates an attack step, which synthesizes adversarial responses that satisfy the current rubric while failing the actual task, with a repair step that revises the rubric to close the exposed gaps while keeping valid criteria intact. The paper reports that rubrics produced this way outperform existing LLM-based rubric generation baselines and lead to more effective downstream rubric-based RL.

arxiv.orgpaperAdded 6 Oct 2026, published 4 Oct 2026🧠 AI

COPEX (COntext Provider EXploitation) is a benchmark that isolates the tool-selecting LLM as the unit of evaluation in Model Context Protocol systems, fixing the surrounding agent stack and varying only the model to measure its intrinsic susceptibility to adversarial context. It covers 25 attack types across 125 scenarios spanning four entry surfaces — model/agent, client, server/tool, and transport — and tests nine models across 3,375 trials, finding a mean attack success rate of 64.4% (58.3%–71.4% by surface); combined input and context scanning cut mean attack success by 49.6% on an eight-attack defense subset. The work separates "system exposure" (attacks succeeding outside the model's observation/control) from "model susceptibility," and the benchmark itself is released publicly.

arxiv.orgpaperAdded 6 Oct 2026, published 3 Oct 2026🛡️ Security

CyTReX is a proposed explainable-AI framework for Distributed Energy Resource (DER) network security that converts edge anomaly-detector alerts into ranked, evidence-traceable threat hypotheses for SOC analysts. It builds a structured "evidence packet" combining edge anomaly detection outputs, cloud-layer attack interpretation, SHAP feature attributions, surrogate decision rules, and Model Context Protocol-enabled cyber threat intelligence enrichment, constraining LLM reasoning so every ranked hypothesis and attack-tree branch is traceable to explicit evidence rather than free-form inference. The authors evaluate five pipeline configurations and show that adding reasoning components improves hypothesis specificity and evidence traceability for SOC triage/investigation.

arxiv.orgpaperAdded 6 Oct 2026, published 3 Oct 2026🛡️ Security

This paper introduces an expert-validated benchmark of 1,042 items spanning ten areas of Colombian law and three question formats (closed multiple-choice, semi-open, open-ended IRAC), used to evaluate 15 proprietary and open-weight LLMs. Closed-question accuracy ranges from 0.577 up to 0.905 (Gemini 3.1 Pro), but free-text factual correctness never exceeds 0.45 for any model, and only about half of the legal norms models cite in free-text answers are correct — the rest are wrong or fabricated. The study also finds a negative correlation (rho = -0.46) between answer relevancy and factual correctness, meaning models sound confident and responsive while frequently being wrong, and argues current LLMs need expert supervision plus source-grounding for reliable legal use.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

The World Embedding Benchmark is a new evaluation suite of 8,000 controlled simulation cases across 80 families (fluid mechanics, solid mechanics, dynamics, optics & electromagnetism), each pairing a rendered video with simulation-derived physical annotations. It tests pre-trained omnimodal embedding models on text-video retrieval, physical-property regression, and multiple-choice video-description classification, finding that current embeddings align poorly cross-modally while lightweight probes can still recover physical quantities, and that continual contrastive training improves alignment but hurts regression. The authors also show retrieval-augmented generation using these embeddings with MiniMax-H3 improves the physical fidelity of generated videos.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

DEPICT is a training-free metric for scoring text-to-image alignment that replaces the fixed-YES assumption of decomposed VQA-based evaluators with an expected-agreement score between image-based and caption-only answers, weighted by how decisively the caption determines each question, then merges this with a holistic score. Evaluated across five benchmarks and eleven vision-language model backbones from three model families, it raises negation accuracy from 19% to 88% and outperforms other training-free metrics while beating fine-tuned evaluators on two of three human-correlation benchmarks.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

This arXiv paper introduces threat-preserving representation sensitivity (TPRS), a metric for how much an agent-security benchmark's attack success rate (ASR) changes purely from cosmetic changes to tool-name wording, with the underlying task, policy, and ground truth held fixed. Testing on Agent Security Bench (ASB), MCPTox and AgentDojo with GPT-5-mini, Claude Haiku 4.5 and GPT-4o-mini, the authors find swings of up to 13 percentage points in ASR from renaming tools as threat-neutral vs. threat-explicit, showing that single-representation ASR scores are not reliable robustness measures and arguing for evaluation across a controlled set of representations instead.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

CorrectGuard is a framework for estimating how accurately black-box security guardrails (jailbreak, harmful-content, prompt-injection and extraction detectors) perform on inputs that cannot be inspected by humans or by the evaluating model itself, using an independent correctness model trained only on labeled "eyes-on" data. The authors test in-context-learning, embedding-based, and fine-tuned correctness classifiers under leave-one-dataset-out evaluation across 13 safety/security datasets and multiple open-weight guardrails, showing up to 25-point macro-accuracy gains and improved risk-coverage tradeoffs (AURC reduced from 0.33–0.44 to 0.17–0.22) for guardrail decision ranking and abstention. The work matters for anyone building auditing or eval pipelines for opaque safety classifiers without privileged access to their internals.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

This paper presents an automated pipeline that uses a vision-language model to annotate leaf midlines in rendered orthographic views of maize point-cloud scans, then applies deterministic geometric algorithms and differentiable NURBS fitting to reconstruct editable, organ-level procedural 3D plant models without manual tuning or species-specific segmentation training. Evaluated on 100 field-grown maize plants from the MaizeField3D dataset, it achieves a median whole-plant Chamfer distance of 5.4 mm and recovers 99.4% of curated reference leaves at IoU ≥ 0.5, outperforming an earlier semi-automated pipeline. It demonstrates that VLM annotations can become reliable quantitative measurements when paired with correcting geometric post-processing, enabling scalable generation of 3D plant assets for high-throughput phenotyping and in silico breeding.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

This arXiv paper replays ground-truth tool calls from the AgentDojo and tau-bench agent benchmarks (without an LLM) to generate benign-by-construction tool outputs, then labels injected outputs via differential replay to evaluate fifteen prompt-injection detectors, including Meta's Prompt Guard 2, plus two task-aware LLM judges, also testing them on BIPIA. It finds detection rankings transfer poorly between benchmarks (the best detector on BIPIA catches only 2% of AgentDojo injections at 1% false-positive rate; a detector catching 72% of AgentDojo injections catches only 15% on tau-bench), while false-positive rates do transfer between the two agent benchmarks, and traces this to mismatches between training data format (e.g., training on short prompt strings vs. tool-output-embedded injections) and deployment context.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🛡️ Security

Persona Guardrail is a production-grade runtime defense framework for customer-facing LLM agents that enforces explicit functional boundaries via synchronous input/output validation against semantic allowlist and blocklist specifications. The authors also introduce PAGE (Persona-Aware Guardrail Evaluation), a benchmark covering benign, adversarial, and out-of-domain user and agent turns; deployed in production, it raises overall accuracy from 85.7% to 95.9% versus a generic LLM-based guardrail, boosts out-of-domain detection from 57.3% to 93.5%, and cuts the false-approved rate from 25.0% to 4.7%, all within latency budget.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🛡️ Security

Evaluates vanilla Double Deep Q-Learning (DDQL), K-means-partitioned mixtures of DDQL experts (K=2, 4, 8) and parameter-matched dense networks for order execution on 5-minute BTC/USDT limit order book data from Binance. No learned configuration significantly beats DDQL on implementation shortfall, and all are worse than TWAP and immediate liquidation in this frictionless replay environment. Ablations show annealed exploration alone removes the collapse to wait-until-forced-liquidation policies (12/100 to 0/100), so expert partitioning appears to mask a training-specification flaw rather than add real benefit. The attribution of the failure mode reverses between 30 and 100 seeds, which makes the paper a case for repeated-seed evaluation in RL.

arxiv.orgpaperAdded 15 Oct 2026, published 2 Oct 2026🧠 AI