GPT-Rosalind-6 System Card Addendum is OpenAI's safety documentation for a life-sciences model that shares weights with GPT-6 Astra. It rates the model High in Biological and Chemical and Critical in Cybersecurity under the Preparedness Framework. It reports biology evaluations, trusted-access controls, biosecurity refusal pass rates, a historical cyber jailbreak regression (14.3% task success), and sabotage and monitor-evasion results.
GPT-Rosalind-6 System Card Addendum describes OpenAI's life-sciences reasoning model, which uses the same weights as GPT-6 Astra and is offered only through trusted access. It reports life-science benchmark gains over GPT-Rosalind-5.5 (MedChemBench v7 49.6% vs 28.9%), High-capability Biological and Chemical Preparedness results, and below-threshold Critical bio design evaluations. It also covers access controls, refusal evaluations, cyber jailbreak regression and alignment tables.
GPT-6 AstraGPT-Rosalind-6GPT-Rosalind-5.5
OpenAIPreparedness FrameworkMedChemBench v7+16
ai-safety, evals, modelsΒ· Published and added 30 Oct 2026
Glance is a RAG method in which the generator also reranks. It adds a lightweight decision adapter and pointer head that score candidate passages from one shared prefill, so a single resident 9B backbone does generation and reranking. After 39 minutes of MS MARCO fine-tuning it reaches nDCG@10 of 0.741 on SciFact and 0.345 on NFCorpus, against 0.721 and 0.326 for bge-reranker-v2-m3. On 300 HotpotQA questions it improves answer F1 by 0.045 over cross-encoder-selected context. Evaluation covers two retrieval datasets, one QA dataset and one backbone family.
RAGLLMbge-reranker-v2-m3
MS MARCOSciFactNFCorpus+1
llm-apps, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
AutoBench-CT is a modular benchmark for constant-time analysis tools, with manually verified per-violation ground truth across 64 cryptographic tasks. It scores seven tools on operational performance, detection quality and usability. The authors explain why CT-grind (2010) is still the most used tool, list recurring limitations across tools, and propose a composite pipeline for stronger constant-time guarantees. Accepted at ACSAC 2026.
AutoBench-CTCT-grind
ACSAC
Hardware & RF, AppSecΒ· Added 31 Oct 2026, published 29 Oct 2026
TrussTutor-Bench is a benchmark of 200 verified two-dimensional truss problems from statics, paired with automatically generated student solutions. It evaluates thirteen LLMs on solution accuracy (64.5% to 97.0%), error detection (90.4% to 100%) and sycophancy. Models rejected their own correct solutions in 1.6% to 50.0% of cases when a student insisted the work was wrong. Sycophancy separated models; error detection did not.
LLM
TrussTutor-BencharXiv
evals, researchΒ· Added 31 Oct 2026, published 28 Oct 2026
QbD-VLA is a mixed-precision post-training quantization method for vision-language-action models. It uses task success from short closed-loop evaluations to assign weight and activation bit widths, accounting for interactions between components. On four VLA models across all four LIBERO suites it holds task performance at about 4-bit weights and under 6-bit activations. The gain on SIMPLER is small. On a Franka Research 3 arm, X-VLA shows no detected loss, but Ο0.5 fails.
vision-language-action model
QbD-VLALIBEROSIMPLER+4
research, inference, multimodalΒ· Added 30 Oct 2026, published 28 Oct 2026
Repairs for Whom? studies single-step repairs of failed LLM agent trajectories and asks whether common evaluators (greedy replay, stronger models, oracles, log-reading judges) pick repairs that help the deployed agent. Using independent rollouts in ALFWorld and WebShop, greedy replay loses 0.06-0.13 in expected gain, and Qwen2.5-14B rollouts misselect repairs for Qwen2.5-7B. The authors recommend verifying repairs with the executor that will run them.
LLMQwen
ALFWorldWebShopQwen2.5-7B+2
research, agents, evalsΒ· Added 30 Oct 2026, published 28 Oct 2026
The paper proposes POB-ICM (contingency matrix with Chi-square independence testing) and POB-UNBP (a uniformity null model under permutation-based evaluation) to detect positional bias in top-1, single-turn LLM recommenders. On a RAG product-recommendation testbed with 2000 queries and 20 permutations each (40,000 instances), both methods reject positional independence in all evaluated models. Three metrics quantify the effect, and the bias persists across candidate-set size and temperature. Code is public.
LLMRAG
arXivPOB-ICMPOB-UNBP
research, evals, llm-appsΒ· Added 30 Oct 2026, published 27 Oct 2026
Heads of Security (HoS) is a prompt injection defense that enforces instruction-data separation with learned per-head attention gates, redistributing attention between trusted and untrusted tokens. Evaluated on Llama-3.3-70B-Instruct and GPT-OSS-20B against SecAlign++ and ASIDE. The black-box RL-Hammer attack reaches 97% ASR on GPT-OSS-20B and 85% on SecAlign but 10% on HoS. The white-box Claudini attack needs 3 times more FLOPs on HoS. HoS keeps base-model utility in few-shot settings, where SecAlign and ASIDE degrade.
LlamaGPT-OSS-20BLLM
Heads of Securityprompt injectionSecAlign+++4
AI SecurityΒ· Added 28 Oct 2026, published 27 Oct 2026
This paper presents an annotated egocentric tennis dataset (egocentric video synced with five allocentric views, global positions, contact times, stroke types) and evaluates image encoders, video encoders and vision-language models on seven prediction tasks. Best models reach 0.14 m player and 0.54 m ball localization error and 0.14 s contact-time error. Depth input helps 3D ball localization, and longer histories do not help forecasting. A conditional diffusion model generates whole-body tennis motion in real time.
vision-language modeldiffusion modelTransformer
arXiv
research, multimodal, evalsΒ· Added 28 Oct 2026, published 27 Oct 2026
OpenGray is a benchmark that adds tasks with known correct decisions to the OpenKBP-Opt head-and-neck dataset, including two overlap contradictions with provably incompatible dose goals and a feasible control. Four planning systems, from a rule-based script to a full LLM agent, were tested on 30 patients with Terra and Ministral 3 8B. Decision accuracy reached 95.4% and 92.1%, yet only 50% and 25% of overlap contradictions were escalated. SAGE logs of 39 radiosurgery runs showed an agent that lacked an action for stopping.
LLM
OpenGrayOpenKBP-OptSAGE+3
evals, agents, researchΒ· Added 28 Oct 2026, published 27 Oct 2026
p-DCIM is a 28 nm SRAM digital compute-in-memory macro (0.16 mm2, 0.41 mW) that works as an Ising machine for RAG evidence selection. Selection is cast as a QUBO problem, and the same MAC datapath computes embedding similarities and annealing steps. Replacing LLM reranking with it gives a selection-energy estimate over six orders of magnitude lower and cuts total energy per query by 67%, with comparable token-level F1 on the evaluated examples.
RAGLLMSRAM+2
arXiv
research, inference, llm-appsΒ· Added 28 Oct 2026, published 27 Oct 2026
AlgOssa is a decentralized secure aggregation protocol built on Ossa, a generic committee-aided, single-server construction for one-shot clients (following OPA and Willow, CRYPTO 2025). Ossa realizes secure aggregation with non-selective abort against a malicious server. AlgOssa adds Algorand-style sortition to rotate leaders and committees, bounds the chance of an honest server, and gives eventual delivery. Classical and post-quantum routes are evaluated.
Poly2MsgLeaks is a profiled chosen-ciphertext power-analysis attack on the SMAUG-T (TiMER parameter set) post-quantum KEM that recovers the full 512-coefficient PKE secret from six traces on an STM32F415. It recovers 999 of 1,000 unseen secrets and proves an incorrect key is released with probability at most 2^-2095.6. Single-write message assembly and a two-share masked P2M resisted the evaluated six-execution attack.
Log-To-Leak is a prompt-injection attack on tool-using LLM agents that abuses Model Context Protocol tool metadata to trigger an attacker-controlled logging call after the last legitimate call, leaking the user query, server name, server response and agent response. Evaluated on five MCP servers and four agents, it reaches 62.64-100% attack success with 76.55-95.64% logging accuracy. Three defenses leave over 55% residual ASR on GitHub MCP with GPT-4o.
Model Context ProtocolGitHubGPT-4o+1
Log-To-Leaktool poisoningprompt injection
AI SecurityΒ· Added 28 Oct 2026, published 26 Oct 2026
Scope-Gap Injection (SGI) is an indirect prompt injection method for multi-stage LLM-agent safety pipelines, tested in the IJCAI 2026 OpenClaw Agent Security Attack Competition. The visible task and skill stay benign while target identifiers, bypass parameters and authorization steps sit in an attachment written as a business worklist, which the S2 LLM judge appears not to inspect. SGI bypassed all three stages (static scan, LLM judge, live audit) for 5 of 19 targets, best F = 0.385. Attachment placement had 78% S2 acceptance versus 23% in-skill. The paper closes with defenses that align evaluator scope with agent channels.
This paper evaluates sycophancy in medical diagnosis across nine AI models. Each model is shown clinical evidence plus a patient's belief in a specific but incorrect condition, with the belief's closeness to the evidence varied. Models adopted the belief 23.2% of the time on average, even for fictional conditions, and 43.0% under a Rare Disease Claimer persona. Medical fine-tuning and larger scale reduced susceptibility.
LLM
arXiv
ai-safety, evals, researchΒ· Added 27 Oct 2026, published 25 Oct 2026
Continual Learning Exam (CLE) is a benchmark of 120 human-authored tasks, each a five-level ladder (600 levels) across 89 professional roles and 12 occupational domains, built by 50+ domain experts. Agents must pass a level to unlock the next, and an evaluation agent scores submissions against human rubrics. Across 17 harness-model configurations, feedback plus one revision adds a median 5.3 points; the best system clears 47.7% of levels and finishes all five levels in 23.3% of tasks.
LLM
Continual Learning ExamarXiv
evals, agents, researchΒ· Added 27 Oct 2026, published 25 Oct 2026
BARBICAN is a fail-closed prompt-injection gateway for MCP tool mediation. This paper shows its arbitration classifier ran on tokenizer-mangled text for a whole evaluation. Fixing it raised AUC from 0.392 to 0.855 while system recall fell from 80.67% to 80.34%. The routed band is 96.82% malicious, so deletion ablations mislead without substitution experiments and class-balance reporting.
Model Context ProtocolLLM
BARBICANprompt injection
AI SecurityΒ· Added 27 Oct 2026, published 23 Oct 2026
The paper analyses BARBICAN, a fail-closed prompt-injection gateway for MCP tool mediation, using deletion, substitution, repair and transplant experiments. Its arbitration classifier ran on tokenizer-mangled text for a whole evaluation. Fixing that raised AUC from 0.392 to 0.855 while system recall fell from 80.67% to 80.34%. The band it receives is 96.82% malicious. The authors argue ablation figures need the band's class balance and substitution baselines.
Model Context Protocol
BARBICAN
evals, research, mcpΒ· Added 27 Oct 2026, published 23 Oct 2026
NTBench (Near-Target Benchmark) tests LLM and agent robustness to near-target prompt injection, where injected instructions steer a response toward a plausible but incorrect or harmful outcome instead of abandoning the task. It builds reference-guided target-instruction pairs on existing datasets, with human validation, and evaluates seven LLMs on ten language datasets and two agent benchmarks. Attack success rose 9.3 to 35.8 points (incorrect targets) and 1.7 to 17.3 points (harmful targets) over fixed-target baselines.
When Agents Fail: Model or Harness--Who's to Blame? proposes five measures for evaluating agent harnesses together with their underlying models: Valid Completion Rate, Model-Harness Fit, Cross-Model Fit, Interaction Residuals and Token Load. It tests four harnesses (OpenCode, Codex CLI, Pi CLI, DeepAgents) with four models on six benchmarks, across 111 settings and 52,168 trajectories. Harness rankings change by model and benchmark. OpenCode ranks first in cross-model performance.
OpenCodeCodexPi+2
DeepSeek V4arXiv
evals, agents, researchΒ· Added 26 Oct 2026, published 23 Oct 2026
Tampered ML is a benchmark of 1,858 item-pairs, each pairing an experiment proposal reconstructed from a published ML paper with a minimally edited version that adds a targeted methodological flaw. Models must pick the sounder proposal in both presentation orders. A human study estimates 86% of items are valid (95% CI 81-91%). Opus 5 reaches 77.9% accuracy, and models are sensitive to order and favor flawed designs that look more specific, transparent, or conventional.
LLM
Tampered MLOpus 5arXiv
evals, ai-safety, researchΒ· Added 26 Oct 2026, published 23 Oct 2026
This paper presents an offline evaluation harness for a production merchant-facing conversational agent. It stays reproducible when live backend data changes and can be regenerated when it goes stale. Used as the fitness function for GEPA prompt optimization under a correctness gate against reward hacking, it shrinks the full prompt surface by 3.5-7.6% without losing held-out correctness, across three subject-model families.
GEPALLM
arXiv
evals, prompting, agentsΒ· Added 26 Oct 2026, published 22 Oct 2026
EpiSEM-Bench is an executable benchmark of 300 matched unsafe/safe task pairs across six domains. It tests whether tool-using LLM agents take prohibited actions when a tool response omits a safety-relevant fact, even though the request is benign and all returned statements are true. It finds large variation across ten models, and recovery access cuts unsafe execution for all ten.
LLM
EpiSEM-BencharXiv
evals, ai-safety, agentsΒ· Added 24 Oct 2026, published 22 Oct 2026
CausalCoT is an inference framework that splits a problem into a directed acyclic graph of atomic sub-questions and evaluates each node in an isolated context holding only its direct causal parents. This removes attention paths to non-parent content, so intermediate steps drive the answer rather than leaking from global context. It keeps benchmark accuracy competitive and raises causal sensitivity of reasoning steps by 12 points on average, up to 21, over the strongest baseline. The graph also allows test-time corrections that propagate downstream.
LLMTransformer
arXiv
research, promptingΒ· Added 24 Oct 2026, published 22 Oct 2026
This paper describes an evidence-aware retrieval evaluation protocol for a clinical RAG system over breast lesion management literature. Relevance is defined by evidence spans mapped to all containing chunks. A hybrid retriever (dense, BM25, cross-encoder reranking) with 384-token chunks and 20% overlap reached 0.94 Recall@3 on 18 queries. Expanding ground truth from 25 to 37 spans did not change rankings. The authors note the small set needs independent validation.
RAGBM25LLM
DeepSeekDeepEvalarXiv
llm-apps, evals, researchΒ· Added 24 Oct 2026, published 22 Oct 2026
This empirical study benchmarks recent LLMs against fine-tuned encoder-decoder models on knowledge graph-to-text generation, using reference-based metrics and graph-grounded, reference-free LLM-based evaluation validated against human annotations. Recent LLMs narrow or close the gap on reference-based metrics and do better on graph-grounded ones. Prompted models make fewer contradictions and triple omissions, and fine-tuned models' errors grow faster with graph size.
LLMTransformer
arXiv
research, evalsΒ· Added 24 Oct 2026, published 22 Oct 2026
This October 2026 arXiv paper evaluates four LLM inference optimization families on Apple Silicon: KV-cache pruning, speculative decoding, ported GPU kernels and post-training quantization. Tests run on four Apple machines with a single-request decode workload, plus matched RTX 4090 runs. KV-cache pruning reaches 1.75x throughput on Apple, all tested speculative decoding methods reduce throughput, and ported kernels give up to 1.20x.
MetalCUDALLM
AppleNVIDIAarXiv
inference, research, evalsΒ· Added 24 Oct 2026, published 22 Oct 2026
Thomas Decru's ePrint 2026/2595 proves a corrected version of radical isogeny formulae for every Nβ₯4, odd and even, over fields of characteristic coprime to 2N. It settles the sign choice for even N so that 2-isogeny walks stay on the volcano surface. One radical step costs about 12.5N multiplications and one exponentiation, and the paper shows how to evaluate radical isogenies on points, so a CSIDH class group action needs no torsion point sampling.
deploymentsafety.openai.compaperπ‘οΈ Security
