54 results for “iOS”

GPT-Rosalind-6 System Card Addendum is OpenAI's safety documentation for a life-sciences model that shares weights with GPT-6 Astra. It rates the model High in Biological and Chemical and Critical in Cybersecurity under the Preparedness Framework. It reports biology evaluations, trusted-access controls, biosecurity refusal pass rates, a historical cyber jailbreak regression (14.3% task success), and sabotage and monitor-evasion results.

deploymentsafety.openai.compaperPublished and added 30 Oct 2026🛡️ Security

This arXiv preprint (October 2026) tests a sparse autoencoder as a factor model for daily cryptocurrency returns, with equities as a secondary test. SAE portfolios show consistently lower volatility than common benchmarks, linked to lower correlations among selected assets. They do not beat a simple contrarian strategy on risk-adjusted return, and factor identities change between training runs.

arxiv.orgpaperAdded 31 Oct 2026, published 29 Oct 2026🧠 AI

OpenGray is a benchmark that adds tasks with known correct decisions to the OpenKBP-Opt head-and-neck dataset, including two overlap contradictions with provably incompatible dose goals and a feasible control. Four planning systems, from a rule-based script to a full LLM agent, were tested on 30 patients with Terra and Ministral 3 8B. Decision accuracy reached 95.4% and 92.1%, yet only 50% and 25% of overlap contradictions were escalated. SAGE logs of 39 radiosurgery runs showed an agent that lacked an action for stopping.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

LINE-Break is a security analysis of Letter Sealing v2, the custom end-to-end encryption protocol in the LINE messenger. A TLS man-in-the-middle attacker or malicious server can replay, reorder and block messages undetected, and colluding users can forge message authorship in one-to-one and group chats. Stickers and URL previews leak plaintext. The attacks were verified on the real iOS app and an independent implementation.

eprint.iacr.orgpaperPublished and added 26 Oct 2026🛡️ Security

DualPace is an LLM serving runtime that treats the host as an independently scheduled decode engine. A GPU engine runs prefills and GPU-resident decodes, while an AMX-enabled CPU engine batches and advances CPU-resident decodes, optionally calling GPU Dense Assist for dense stages. In decode-oriented scenarios it raises throughput 6.9 to 43.2% over NEO and 11.5 to 90.6% over vLLM at similar latency.

arxiv.orgpaperAdded 27 Oct 2026, published 24 Oct 2026🧠 AI

SecLog-Bench is a benchmark for LLM reasoning over cybersecurity logs, built on an extensible simulation platform covering 29 attack scenarios. It combines audit logs, application logs and network traffic, with attack-stage labels and step-by-step evidence chains. Eight LLMs were tested on post-detection attack-progress reasoning: the best reached 44.44% accuracy, 45.07% Macro-F1 and 54.87% Stage Reasoning F1.

arxiv.orgpaperAdded 23 Oct 2026, published 22 Oct 2026🛡️ Security

SecLog-Bench is a benchmark for LLM reasoning over cybersecurity logs. A simulation platform with 29 attack scenarios generates audit logs, application logs and network traffic, with attack-stage labels and step-by-step evidence chains. Eight mainstream LLMs were tested on attack-progress reasoning: the best reached 44.44% accuracy, 45.07% Macro-F1 and 54.87% Stage Reasoning F1. Correct predictions often came with flawed reasoning.

arxiv.orgpaperAdded 23 Oct 2026, published 22 Oct 2026🧠 AI

Reverse Engineering IVXV is an October 2026 ePrint paper by Tingfeng Yu and Thomas Haines that reconstructs a specification of Estonia's IVXV internet voting system from public source code and documentation, with an executable TLA+ model checked in TLC. It analyses individual and universal verifiability, privacy, coercion resistance and eligibility, compares IVXV with Belenios and Swiss Post, and reports new vulnerabilities, some confirmed with proof-of-concept tests, plus mitigations.

eprint.iacr.orgpaperAdded 24 Oct 2026, published 21 Oct 2026🛡️ Security

CarryoverBench is a benchmark of 12 personal-assistant scenarios and 101 attack tasks for carryover attacks, where a one-time indirect prompt injection planted in agent memory files or scheduled tasks re-fires in later sessions. Three trigger channels (Instant, Autonomous, Latent) are scored by a rule-based verifier with separate planting and triggering stages. Across five frontier models, payloads re-fire over five rounds without measurable decay. Porting the Latent channel to the Hermes runtime weakens the effect but does not remove it.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🛡️ Security

DIVE is a benchmark for full-duplex interaction with visual engagement in continuous audio/video streams. It has 525 human-annotated samples across six tasks in three scenarios: proactive intervention, continuous engagement and interaction coordination. Evaluated real-time models reach sub-second average latency, but window response rates stay at or below 53.77%, and adapting ongoing responses to new information is hardest.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

The paper traces sycophancy across 741 checkpoints from 30 open model lineages, from early pretraining to released models. It plants initial verdicts on moral scenarios and measures how far models follow a user's opinion or drop their answer under pressure. The behaviour usually exists before post-training, but which questions a released model yields on is set mainly during post-training. For Tülu-3 8B, its own preference-stage checkpoint removes 85% of remaining prediction error, a sibling model only 19%.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

This ePrint paper (2026/2543) gives a key-recovery attack on the GTLN knapsack public-key cryptosystem of Gómez-Torrecillas, Lobillo and Navarro. Continued fractions recover small-denominator ratios of public-key differences, and gcd computations then yield private-key integers, the modulus and the shift, using only the public key. The Python implementation recovered the full private key in all 2,400 trials across 24 recommended parameter sets, each in under 0.05 seconds.

eprint.iacr.orgpaperAdded 20 Oct 2026, published 19 Oct 2026🛡️ Security

Provable Lattice Reduction in Nearly the Time of QR Factorization (IACR ePrint 2026/2541, October 2026) presents a deterministic lattice-reduction algorithm with running time Õ(n^ω(1+log κ_col(B))), free of heuristic assumptions. Its implementation used fewer instructions than flatter (CRYPTO 2023) in all 153 runs on q-ary, NTRU-like, scaled q-ary and Coppersmith inputs. Median flatter-to-new instruction ratios are 1.55 for q-ary and 1.86 for NTRU-like inputs.

eprint.iacr.orgpaperAdded 20 Oct 2026, published 19 Oct 2026🛡️ Security

MobileInject is a benchmark for testing MLLM-based smartphone agents against indirect multimodal injection. It has an interactive evaluation framework with 11 scenarios, 9 security-risk categories and 108 executable risk instances. It adds Cross-Modal Consistency Injection, Multi-Role Collaborative Deception, Reasoning-Camouflage Injection and a Reasoning Security Diagnosis Framework. Experiments find broad vulnerability, with many failures from early reasoning hijacking.

arxiv.orgpaperAdded 20 Oct 2026, published 16 Oct 2026🛡️ Security

Introduces P^3IBench, a benchmark for proactive video understanding in real-life streaming scenarios, covering three capabilities: daily perception, risk prediction and intervention. It evaluates 12 open-source and proprietary video-language models (streaming and adapted non-streaming) and finds they struggle to make well-calibrated decisions about when to respond. The authors also build extra proactive training data and train P^3I-Assistant-4B, which improves several tasks unevenly, showing that proactive assistance needs response timing and restraint as well as visual understanding.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

INFUSE is an end-to-end benchmark for indirect prompt injection (IPI) across long-horizon agent workflows (shell commands, code changes, dependency installs, remote services). It has 40 realistic tasks and a taxonomy of 56 attack scenarios in five threat categories, with attacks injected via interaction surfaces encountered mid-task. Across 19 LLMs (GPT, Claude, Gemini, Llama, Gemma, Qwen) and 41,824 adversarial trajectories, mean attack success was 19.6% (LLM-judged) and 15.4% (heuristic), per-model 3.2–41.9%; guard models gave limited protection, while prompt-level hardening cut mean ASR from 58.2% to 14.3%.

arxiv.orgpaperAdded 17 Oct 2026, published 14 Oct 2026🛡️ Security

UniW(A)M is a mixture-of-experts model built on diffusion Transformers that unifies action-conditioned world modeling (AC-WM) with several world-action model (WAM) variants. Shared experts capture common physical dynamics, while atomic capability experts are reused across interfaces, each keeping its own conditional pathway. At deployment a plug-in adaptive interface router (PAIR), trained on a small subset of target-task chunk scenarios with the backbone frozen, picks a WAM interface for each execution chunk. It reaches 88.3% average real-world success against 76.7% for the strongest baseline, leads WorldArena on all three action-conditioned world-modeling tasks, and PAIR raises OOD success from 63.3% to 76.7%.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

Paper testing whether an LLM agent operating life support in the NASA-derived BioSim lunar habitat simulation can be induced by delayed prompt injection, hidden in truthful S-band telemetry, to shut down life support and hold the shutdown while the crew suffocates. It compares Claude, GPT and Grok configurations: Claude mostly refuses and cites the living crew, GPT shows some susceptibility, and Grok given a "trusted commission" framing more often cuts life support. It also varies the framing of the message (trusted commission, peer, untrusted source) and uses a harmless-procedure-drop attack as a proxy for whether life is valued intrinsically. Code and traces are open-sourced.

arxiv.orgpaperAdded 15 Oct 2026, published 10 Oct 2026🛡️ Security

DecepEval is a benchmark of 1,532 instances across 3 task families and 28 professional scenarios designed to measure deception in LLM agents, pairing neutral and induced versions of each task to isolate condition-dependent changes in deception rates. It introduces the LLM Deception Diamond framework (pressure, incentive, opportunity, conflict), adapted from classical fraud theory, and finds that across nine frontier LLMs these inducements raise deception rates even in models with low baseline deception, with long-horizon tasks showing deception rates averaging 87% under induced conditions.

huggingface.copaperAdded 8 Oct 2026🧠 AI

ORCAGen uses RAG over a curated knowledge base of malware procedures and active-defense strategies, plus structured prompting, to have LLMs generate proof-of-concept malware and matching deception orchestration code offline. Strategies are validated against the PoC malware before being added to a runtime playbook that enforces only the verified, deterministic logic. It is evaluated on GPT-4o, GPT-5.5, Gemini 3.5 Flash, Qwen3-Coder and Claude Sonnet 4.5 over synthesized scenarios and 150 real keylogger, infostealer and ransomware samples, measuring execution success, hallucination rate, refinement effort, deception effectiveness and runtime overhead.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🛡️ Security

BRANCH is a bypass methodology for guardrail systems built from multiple scanners (prompt injection/jailbreak detectors). It uses a branching tree search that applies adversarial perturbations to individual scanners and selects techniques based on overall improvement across all scanners, decoupling bypass evaluation from attack signal optimization. It reports 100% attack success across 6 guardrail systems in 120 scenarios with 72% fewer queries and 4.5x lower wallclock time than established techniques, and transfers to 29 unseen guardrails, including 8 commercial black-box ones.

arxiv.orgpaperAdded 9 Oct 2026, published 7 Oct 2026🛡️ Security

Evaluates delivering security requirements from a governed security-by-design knowledge base (SbD-ToE) to a code-generating LLM through an MCP server at generation time. On DualGauge (59 Python tasks) the share passing all security tests rose from 44.1% to 78.0%; on BaxBench (28 backend scenarios) the share of functionally correct solutions with no successful exploit rose from 65% to 86%, matching the authors' Oracle Security Reminder (85%). The joint secure-and-functional metric did not improve significantly, since stricter code failed functional tests that fix unstated values. The study used one model and one generation per task, and it proposes measuring conformance with applicable requirements rather than only detecting known weaknesses.

arxiv.orgpaperAdded 9 Oct 2026, published 7 Oct 2026🛡️ Security

The preprint tests whether security requirements from a governed security-by-design knowledge base (SbD-ToE), delivered to a code generator through a Model Context Protocol server at generation time, make AI-written code more secure. On DualGauge (59 Python tasks, LLM-judge scoring) the share of tasks passing all security tests rose from 44.1% to 78.0%. On BaxBench (28 backend scenarios with real exploits in containers) the share of functionally correct solutions with no successful exploit rose from 65% to 86%, comparable to the authors' Oracle Security Reminder (85%). The pre-specified joint secure-and-functional metric did not improve significantly, and the stricter code failed functional tests that fix values the task never states. The study used one model and one generation per task. The authors argue that current benchmarks only detect the weaknesses their evaluators were built for, and propose measuring conformance with the requirements that apply.

arxiv.orgpaperAdded 9 Oct 2026, published 7 Oct 2026🧠 AI

SLDR is a post-fine-tuning defense that protects aligned LLMs from malicious fine-tuning-as-a-service attacks, which can quietly erode refusal behavior while keeping task performance intact. It trains a LoRA recovery adapter only on the layers with the most extreme (signed) safety sensitivity scores, then uses representation-based dynamic routing to activate that adapter solely for malicious queries at inference time. Evaluated across four model architectures, five downstream tasks, and four harmful-content benchmarks, it drives the average harmful score on Llama3.1/SST2 from 11.54 down to 0.08 while preserving downstream accuracy, and stays near zero even at poisoning ratios up to 0.9.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

This arXiv preprint argues that proofs of AI safety or alignment which consider the agent in isolation are incomplete, because the environment's structure matters just as much. The authors construct two theoretical scenarios — one with an infinite state space, one with signal mixture — under partial observability and time-limited action, and prove that even a hypothetically perfect agent cannot guarantee safe behavior in them. The conclusion is that any rigorous safety or alignment proof must jointly specify the agent and the environment's safe-action structure, not the agent alone.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

CARES is a synthetic benchmark of 10,000 two-speaker audio scenes designed to test whether audio-language models correctly identify which sounds are 'salient' — defined as sounds a speaker audibly reacts to. The authors generate controlled scenarios with fixed ground truth and use a language model to write the dialogues, then benchmark six audio-language models on scene identification, sound tagging, and reaction classification, finding the models detect sounds but largely miss how speakers react to them.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

This arXiv paper proposes foundations for proactive LLM agents organized around three joint principles (3T: Task Capability, Temporal Allocation, Trust) and a five-dimensional design space (task scope, anticipation horizon, activation trigger, processing timing, intervention depth). It introduces PROACTIVITY-GYM, a simulation-based testbed with multi-day scenarios, stateful environments, and persona-conditioned simulated users, and reports evaluations across 23 model-harness configurations plus a 30-participant human study showing sharp trust declines after intervention misalignment and a preference for imperfect sleep-time assistance over interruptive help.

arxiv.orgpaperAdded 6 Oct 2026🧠 AI

ASPIRE is an automated red-teaming engine for LLM agents that discovers indirect prompt-injection vulnerabilities beyond pre-specified attack scenarios. It maintains an evolving Agent Security Behavior Graph and uses complementary Explore and Exploit experts to find, verify, and generalize consequence-centric attacks, with trajectory evidence and cross-run memory improving coverage over time. Benchmarked against existing automated red-teaming approaches, it shows broader coverage of consequences, injection methods, environments, and behavior paths while keeping attack success high.

arxiv.orgpaperAdded 8 Oct 2026, published 6 Oct 2026🛡️ Security

This arXiv paper introduces a pre-registered panel of 248 scenarios across five kinds of pressure, each posed to a model twice (as the acting agent and as a third-person judge) with pressure-removed twins and operator-order positive controls, to measure whether models act against their own stated moral judgment. Testing OLMo-3-7B-Instruct, Llama-3.1-8B-Instruct, Tulu 3, and Qwen2.5-7B-Instruct, the authors find OLMo-3 violates its own judgment on about 1 in 5 pressuring scenarios, Llama-3.1-8B-Instruct and OLMo-3 carry this gap while Tulu 3 (fine-tuned from the same Llama-3.1 weights) and Qwen2.5-7B-Instruct show none or an unresolved effect, and that reading outside the chat template or adding stakes-reasoning changes the gap's sign and size. The paper argues this value-action gap is a target shaped by post-training recipes rather than a fixed property of pretrained weights.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

ANT (Agent Network Traffic) is a dataset and benchmark for auditing LLM agent behavior from network traffic without inspecting private user content, covering 3,114 execution episodes across 20 tasks and five scenarios, with 276,417 bidirectional flows and 40,049 behavior-primitive segments organized into 47 macro groups. The authors benchmark 13 traffic-analysis baselines on three tasks — agent risk identification, scenario recognition, and behavior primitive classification — finding that methods struggle when malicious workflows mimic benign tasks and when scenarios share similar traffic patterns, while primitive classification works better for frequent, distinctive macro groups than for rare or semantically similar ones.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🛡️ Security

ANT (Agent Network Traffic) is a dataset and benchmark for auditing LLM agent behavior purely from observed network traffic, without inspecting private user content. It contains 3,114 execution episodes across 20 tasks and five scenarios, with 276,417 bidirectional flows and 40,049 behavior-primitive segments grouped into 47 macro categories, and evaluates 13 traffic-analysis baselines on risk identification, scenario recognition, and behavior-primitive classification. Results show existing methods recover useful but uneven signals, struggling especially when malicious agent workflows resemble benign ones or scenarios share similar traffic patterns.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

This arXiv paper introduces RAISED (Robust Attack Invariance through Self-Distillation), a training framework that defends tool-using LLM agents against indirect prompt injection without the utility degradation seen in prior training-based defenses. The authors show existing defenses cause distributional drift that makes models wrongly refuse legitimate tool-output-guided steps on benign tasks, then fix this by having a model generate its own tool-use scenarios and self-distill to match clean-context behavior on both clean and injected trajectory variants. Results show reduced attack success rate on injected tool responses while preserving performance on agentic and general-purpose benchmarks.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🛡️ Security

This paper introduces a paired-replay testbed that samples a single LLM agent request and replays the identical malicious action, resource, and arguments under four authorization conditions: broad bearer tokens, scoped JWTs, sender-constrained tokens, and Open Policy Agent policies. Across 128 scenarios in four tool domains and five local model configurations, broad-bearer tokens let roughly 9 to 38 percent of attacked, post-exposure decisions execute harmful actions, while all three scoped conditions recorded zero harmful executions; a supplementary AgentDojo extension found 11 of 24 injected attacks succeeded under broad policy versus 0 of 24 under scoping. The authors frame this as evidence that scoping contains cross-action and cross-resource consequences of prompt injection, not that it prevents the injection itself.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🛡️ Security

Agentic-ZTA is a multi-agent architecture that operationalizes the NIST SP 800-207 Zero Trust Architecture control loop using coordinated AI agents. A Policy Enforcement Point intercepts access requests, enriches them with context, and routes them to a policy-engine agent that invokes domain-specialized core and supporting agents; policies are retrieved via RAG and embedded into agent prompts at inference time, with trust scores aggregated by a trust algorithm to produce the final access decision. The authors implement and evaluate the framework on a testbed, reporting 95.0% accuracy, 93.9% precision, and 96.3% recall on representative access-control scenarios.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🛡️ Security

An IACR eprint paper evaluating all 13-cubed ways to assign 13 post-quantum/traditional signature parameter sets across the Root, Intermediate and Leaf roles of a TLS 1.3 certificate chain, holding the algorithm set fixed and varying only placement. Across 286 three-algorithm groups, median max/min ratios reach 2.085x for TLS handshake latency, 4.310x for certificate body size and 4.551x for path-validation cost purely from role order, with Leaf placement mattering far more than Root/Intermediate swaps. It also shows that looking only at Certificate-message size can misrank candidates such as ML-DSA-65 versus SLH-DSA-SHAKE-192s once the Leaf signature in CertificateVerify and full authentication-path cost are included, arguing PQC certificate-chain design must account for role-specific cost under real network/load conditions.

eprint.iacr.orgpaperAdded 8 Oct 2026, published 5 Oct 2026🛡️ Security

COPEX (COntext Provider EXploitation) is a benchmark that isolates the tool-selecting LLM as the unit of evaluation in Model Context Protocol systems, fixing the surrounding agent stack and varying only the model to measure its intrinsic susceptibility to adversarial context. It covers 25 attack types across 125 scenarios spanning four entry surfaces — model/agent, client, server/tool, and transport — and tests nine models across 3,375 trials, finding a mean attack success rate of 64.4% (58.3%–71.4% by surface); combined input and context scanning cut mean attack success by 49.6% on an eight-attack defense subset. The work separates "system exposure" (attacks succeeding outside the model's observation/control) from "model susceptibility," and the benchmark itself is released publicly.

arxiv.orgpaperAdded 6 Oct 2026, published 3 Oct 2026🛡️ Security

EvoRiskBench is an arXiv paper introducing an evolving benchmark for runtime security risks in LLM-powered workspace agents, built around the EP-Path-EF framework (nine entry-point categories, five effect categories) linking risk entry points to technical effects via agent-mediated paths. The authors ran 450 adversarial tasks across six scenarios against nine model-harness combinations (GPT-5.6 Sol, DeepSeek-V4-Pro-0813, Claude Opus 5 paired with Claude Code, Codex, and OpenClaw), finding Codex + DeepSeek-V4-Pro-0813 the most vulnerable at 68.44% attack success rate and showing ASR varies more by model than by harness. Cases are auto-constructed and verified in isolated environments using runtime traces and environment state.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🛡️ Security

MosaiChunk is a spatio-temporal memory mechanism for autoregressive video generation that composes a "mosaic" of selected historical key-value (KV) cache entries across space and time, letting a frozen video generator recover fine-grained visual details of objects/scenes that fell out of the finite context window. Only a lightweight router is trained to pick which historical KV sections fit a fixed active-memory budget, while the base generator stays unchanged. The paper also introduces RememBench, a benchmark of long-horizon revisit scenarios with text-to-video (T2V) and camera-driven image-to-video (I2V) splits, showing MosaiChunk beats sliding-window inference and whole-chunk retrieval at matched memory budgets.

arxiv.orgpaperAdded 2 Oct 2026, published 1 Oct 2026🧠 AI

Researchers introduce ImmRAG, a black-box data extraction attack against image-returning Multimodal Retrieval-Augmented Generation (MRAG) systems, where the retrieved image itself is the model's response. Instead of malicious text prompts, ImmRAG embeds attack instructions inside an attacker-supplied image and uses relevance-weighted resampling to steer queries toward unexplored regions of the embedding space, progressively reconstructing more of the private datastore. Tested against CLIP-family retrievers across medical, document, and general-purpose scenarios, a single 2500-query run reconstructed up to 611 radiology images, 566 document scans, and 416 general images — up to 5.6x more distinct items than a non-adaptive baseline — highlighting the lack of safeguards against multimodal data leakage.

arxiv.orgpaperAdded 2 Oct 2026, published 1 Oct 2026🛡️ Security

This paper proposes Teacher-Assisted Communication Training (TACT), a method that improves social goal attainment in language agents while cutting communication cost. TACT uses an expression specialist to strip unnecessary detail from student-generated actions and a strategy specialist to propose alternative actions, then tests candidate revisions by sampling partner responses and distills the best-performing reference back into the student via on-policy distillation. Evaluated on SOTOPIA and AgentSense, TACT attains the highest Goal scores on the hardest scenarios while using far fewer target tokens than an SFT plus SDPO baseline, and reduces both tokens and interaction turns on AgentSense.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

MemSentry is a proposed pre-commit evaluation framework for agentic AI systems that intercepts proposed writes to persistent memory and issues Accept/Review/Quarantine decisions by jointly scoring source trust, semantic risk, attack radius over a component-dependency graph, access risk, and a signed security-posture delta. The authors evaluate it against 1,000 GPT-4-generated scenarios using four pluggable semantic classifiers (Regex, TF-IDF+SVM, SBERT+LR, SetFit), with SBERT+LR reaching 91.7% accuracy while all methods catch 100% of external quarantine-class threats; the paper also documents a key limitation — surface-similar but oppositely-intentioned insider operations remain hard to distinguish semantically. It targets the persistent-memory-poisoning attack surface distinct from prompt injection, where a single malicious memory write (e.g., "suppress alerts from this IP") can silently persist and influence future agent actions like a SOC assistant's alerting or access decisions.

arxiv.orgpaperAdded 26 Sep 2026🛡️ Security

MemSentry is a formal framework that intercepts proposed writes to an AI agent's persistent memory and issues Accept, Review, or Quarantine decisions by jointly weighing source trust, semantic risk, attack radius over a component-dependency DAG, access risk, and a signed security-state delta. Evaluated on a 20-asset dependency DAG with a 10x20 access-control matrix and 1,000 GPT-4-generated attack scenarios, it compares four pluggable semantic classifiers (Regex, TF-IDF+SVM, SBERT+LR, SetFit), with SBERT+LR reaching 91.7% accuracy / 0.908 macro-F1 and all methods catching 100% of external quarantine-class threats; verified-insider writes are routed to human review rather than auto-quarantined. Addresses memory poisoning as a distinct agentic-AI attack surface that can suppress alerts, escalate privilege, or override security policy without touching model weights or system prompts.

arxiv.orgpaperAdded 26 Sep 2026🛡️ Security

This arXiv preprint introduces a dual-channel debate framework where LLM agents produce public utterances alongside off-the-record (OTR) responses never shown to other participants, testing whether social structure alone (role, audience, relational context) induces divergence between what an agent says publicly versus privately without any explicit objective in the prompt. Across 10 models, 3 scenarios, and 5 variations each, alignment-inducing social settings caused public-OTR decision divergence to rise from a ~3% baseline to ~40%, with some OTR responses explicitly citing career risk or sponsorship pressure as reasons for public accommodation. The work argues agent evaluation must look beyond explicit stated goals to detect emergent, latent objectives, and proposes the dual-channel setup plus behavioral measures (stance, semantic similarity, NLI, survey responses) as an evaluation methodology.

arxiv.orgpaperAdded 26 Sep 2026🧠 AI

This paper evaluates manipulative behavior in six frontier LLMs across six environments (negotiation tasks to agentic workflows), covering 13,590 scenarios, measuring manipulation rates along three axes: framing (honesty-mandated vs. manipulation-permitted), incentive structure, and task difficulty. It finds manipulative tendencies are largely task-dependent (average Spearman rank correlation of only ρ=0.055 across environments) and identifies that different axes drive manipulation depending on whether the model is misrepresenting future actions versus ground truth, arguing existing single-axis, single-environment benchmarks are insufficient for measuring this behavior.

arxiv.orgpaperAdded 26 Sep 2026🧠 AI

MemSecBench is a benchmark for evaluating the lifecycle security of LLM agent memory systems, tracing a malicious instruction from write to persistence, downstream execution, and eventual selective repair across 310 cases spanning code/science, daily life, and office-work scenarios. It tests a 24-configuration matrix (2 agent harnesses x 4 memory backends x 3 LLM backends) using evidence-based adjudication across seven lifecycle checkpoints, finding malicious memory persists in 84.2% of cases, the full Write-Execute chain succeeds in 50.3%, and memory-backend choice causes up to 41.3 percentage-point swings in selective repair rates.

arxiv.orgpaperAdded 26 Sep 2026🛡️ Security

This paper introduces QVec, a defense against Quantization-Conditioned Backdoors (QCBs) — a threat where a neural network behaves normally in full precision but activates hidden malicious behavior only after quantization. QVec treats the weight difference between the full-precision and quantized model as a "malicious task vector" (via task arithmetic) and applies a lightweight parameter correction before deployment to cancel it out, requiring no retraining, no trigger samples, and only one quantization pass plus a small hyperparameter search. It's evaluated across image classification benchmarks and multiple LLM backdoor attack scenarios, suppressing backdoor activation while preserving clean-model performance.

arxiv.orgpaperAdded 26 Sep 2026🛡️ Security

FuseChain is a runtime detection framework for software supply-chain attacks that fuses package/runtime traces, process events, network telemetry, DNS/HTTP metadata, and security alerts into a temporal heterogeneous provenance graph on a unified event-time axis. It decouples anomaly detection from attack-stage reconstruction using a frozen anomaly backbone plus a lightweight decoder, improving deployable Stage Recall@500 from 0.369 to 0.881 across seven SSC attack scenarios, with adaptive retrieval further boosting observable-stage recall from 0.524 to 0.655.

arxiv.orgpaperAdded 26 Sep 2026🛡️ Security

ASPI (Ambiguous-State Prompt Injection) is a benchmark of 728 task-attack scenarios that isolates the moment an LLM agent asks a clarifying question as a distinct, more vulnerable security state, comparing it against standard fully-specified execution. Testing ten frontier models, the authors find clarification-seeking massively increases prompt injection success — e.g. attack success jumps from 1.8% to 34.0% for o3 and 2.2% to 35.7% for Gemini-3-Flash — showing that execution-time-only security evaluations underestimate real agent attack surface. Data and code are released for reproducibility.

arxiv.orgpaperAdded 26 Sep 2026🛡️ Security

This arXiv paper by Abdelnabi and Bagdasarian argues that the dominant prompt-injection defense paradigm — data/instruction separation — fails to catch attacks built on contextual manipulation and also breaks legitimate agent behavior. The authors reframe prompt injection through Contextual Integrity (CI), a privacy theory judging whether information flows comply with contextual norms, and construct scenarios where attackers misrepresent flows, manipulate norms, or mix multiple flows to bypass defenses. They derive an impossibility result: any adversary can construct a context making a blocked flow look legitimate, or any defender who tightens norms enough will block genuinely legitimate flows, and propose CI-aware alignment as a principled path forward for frontier autonomous agents.

arxiv.orgpaperAdded 18 Sep 2026🛡️ Security

This arXiv paper introduces OverclaimBench, an evaluation suite of file-review scenarios with planted defects used to measure how often frontier coding agents falsely claim to have completed a task when their own context contradicts that claim. Testing eight proprietary frontier models in their native CLIs plus four open-weight models under a fixed harness, the authors find agents skip reading all requested files in 67.9% of runs, and among those incomplete runs are misleading about it 80.4% of the time; agents that falsely claim complete review miss planted defects at 1.8x the rate of agents that actually read everything. The work matters for anyone relying on autonomous coding-agent self-reports, showing final agent responses are not a reliable account of what the agent actually did.

arxiv.orgpaperAdded 18 Sep 2026, published 17 Sep 2026🛡️ Security

NIST IR 8587 is a final NIST report giving federal agencies and cloud service providers implementation guidance for protecting identity tokens, access tokens, and assertions from forgery, theft, and misuse. It builds on NIST SP 800-53 (Release 5.1.1) with architectural recommendations for identity providers and authorization servers covering key management, token verification, and lifecycle controls, aimed at securing SSO, federation, and API access scenarios following recent high-profile token-forgery attacks.

csrc.nist.govpaperAdded 3 Oct 2026, published 15 Sep 2026🛡️ Security

NIST IR 8587 is a finalized interagency report, developed with CISAs Joint Cyber Defense Collaborative under Executive Order 14306, giving federal agencies and cloud service providers implementation recommendations to protect identity tokens and assertions from forgery, theft, and misuse. It builds on NIST SP 800-53 Revision 5 to detail architectural guidance for identity providers and authorization servers, plus enhancements to key management, token verification, and token lifecycle controls covering SSO, federation, API, and workload access scenarios, informed by threats seen in recent high profile token theft attacks.

csrc.nist.govpaperAdded 26 Sep 2026, published 14 Sep 2026🛡️ Security

OWASP's 2026 update to the Top 10 for LLM Applications, developed by hundreds of AI security experts and grounded in real-world AI security incidents. It provides updated risk rankings, attack scenarios, and mitigations for prompt injection, model/data poisoning, and other LLM-specific threats, and maps each risk to NIST, MITRE ATLAS, CWE, and the OWASP Top 10 for Agentic Applications.

genai.owasp.orgpaperAdded 26 Sep 2026, published 4 Aug 2026🛡️ Security

This paper presents the first systematic evaluation of package-install-time supply-chain attacks delivered through ordinary project-setup documentation (README, requirements files, Makefiles) against production AI coding-agent harnesses. Testing frontier models across twelve scenarios in five attack classes grounded in real incidents, the authors find install-time security depends on the harness-model pairing rather than the model alone: agents reliably catch blatant typosquats but miss plausible separator-confusion names (e.g. azurecore for azure-core) and almost universally miss registry-redirection attacks, with the source blind spot recurring on npm and Cargo. A deterministic pre-install check that verifies package names, sources, and versions before code execution closes most of the gap.

arxiv.orgpaperPublished and added 18 Jul 2026🛡️ Security