The paper treats a benchmark as an empirical measure over item difficulty and formalizes two failure modes: measure mismatch (benchmark difficulty distribution differs from the intended one) and coverage gaps (uncovered difficulty regions that hide localized training progress). It proposes two diagnostic metrics with theoretical guarantees and analyzes 25 popular benchmarks across 7 domains using estimated item difficulties. It also introduces BenchFill, a difficulty-targeted item rewriting algorithm that reduces local coverage gaps.
WorkFactory is a framework that builds coherent, executable professional tasks for training working agents. It plans an information support graph from professional intents, then uses probe executions to find and repair missing or inconsistent task information. The authors built 112,686 tasks across 612 occupations and fine-tuned Qwen models at 4B, 27B and 35B-A3B (WorkPilot). WorkPilot-35B-A3B rose from 879 to 1565 Elo on GDPval-AA v2.1.
GPT-Rosalind-6 System Card Addendum is OpenAI's safety documentation for a life-sciences model that shares weights with GPT-6 Astra. It rates the model High in Biological and Chemical and Critical in Cybersecurity under the Preparedness Framework. It reports biology evaluations, trusted-access controls, biosecurity refusal pass rates, a historical cyber jailbreak regression (14.3% task success), and sabotage and monitor-evasion results.
RADIOlogy is a training-free, single-pass zero-shot anomaly detection method that uses multi-head attention pooling in vision-language models. It scores anomalies by contrasting average similarity between vision features and sets of normal and abnormal prompts, with no windowed execution. RADIO agglomerative features drive a dense refinement pipeline for pixel-level segmentation, and the method is applied to medical images.
CLIPRADIOvision-language model
RADIOlogyarXiv
research, multimodalΒ· Added 31 Oct 2026, published 29 Oct 2026
VeriCUA is a platform-agnostic framework that verifies task completion by computer-using agents through agentic exploration. The verifier inspects context to build a task rubric, then picks GUI or code checks for unresolved requirements and combines findings under explicit evidence rules. It works online and offline across desktop, mobile and web, and agrees better with reference labels than the baselines, with the largest offline gains on hard cases.
vision-language modelVeriCUA
arXiv
agents, evals, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
PhysCount is a benchmark of 546 real-world images and 4,595 human-written questions in 651 target-specific groups. It tests whether multimodal LLMs can separate visible appearances (direct view, reflections, glass) from the physical entities behind them. Across 40 MLLMs, counting appearances is much easier than linking them to entities, and the best model answers every question correctly in only 13.36% of groups.
vision-language model
PhysCountarXiv
evals, multimodal, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
Glance is a RAG method in which the generator also reranks. It adds a lightweight decision adapter and pointer head that score candidate passages from one shared prefill, so a single resident 9B backbone does generation and reranking. After 39 minutes of MS MARCO fine-tuning it reaches nDCG@10 of 0.741 on SciFact and 0.345 on NFCorpus, against 0.721 and 0.326 for bge-reranker-v2-m3. On 300 HotpotQA questions it improves answer F1 by 0.045 over cross-encoder-selected context. Evaluation covers two retrieval datasets, one QA dataset and one backbone family.
RAGLLMbge-reranker-v2-m3
MS MARCOSciFactNFCorpus+1
llm-apps, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
On-policy state training is a method for training chain embedding retrievers for multi-hop RAG on the states the encoder itself reaches, including those built on its own wrong retrievals. Each state is labeled with the earliest missing gold passage, other missing gold passages are masked, and updates are contrastive and mixed with teacher-forced states. It retrieves complete evidence more often than teacher-forced training and graph-based systems, and transfers to harder long-chain HotpotQA and 2WikiMultiHopQA variants the authors built.
RAGon-policy state trainingcontrastive learning+1
HotpotQA2WikiMultiHopQAarXiv
research, llm-apps, trainingΒ· Added 31 Oct 2026, published 29 Oct 2026
CEDAR (Clinical Evidence-Driven Agent for Risk-prediction) is an evidence-citing, self-auditing LLM pipeline for predicting pediatric diabetes outcomes at 24 months from 18 months of records. On 2,489 youth with type 2 diabetes and 4,018 with type 1, pooled AUROC was 0.815 versus 0.656 to 0.700 for trained ML in T2D, and 0.756 versus 0.603 to 0.631 in T1D. Only 4 of 70,246 cited values mismatched the chart. Two pediatric endocrinologists matched its predictions in about nine of ten cases.
LLM
CEDARarXiv
research, agentsΒ· Added 31 Oct 2026, published 29 Oct 2026
EpiCoach is an agentic system that treats one cross-country mountain biker as an N-of-1 experiment. Claims live in an evidence-gated hypothesis registry, and each must make a probabilistic prediction before a session. Predictions come from a digital twin fitted to heart rate, GPS and altitude. In a replay over eighteen months the twin beat standard forecasters on heart rate but not segment time, and the agent loop matched a static model. In the first three live weeks, the ledger sealed 20 predictions and scored 14, and a safety officer vetoed the agent twice. The registry, benchmark and protocol are released.
LLMBayesian experimental designdigital twin+1
EpiCoacharXiv
agents, research, evalsΒ· Added 31 Oct 2026, published 29 Oct 2026
CORAL (Cognitive Resource Self-Allocation) is an agent-callable working-memory toolset that lets an LLM agent checkpoint task progress and verified facts, then reset its cluttered context and resume from a clean state. A Multi-episode Agentic Reinforced Policy Optimization algorithm trains the checkpointing. With an 8B model it reaches 42.8% average accuracy on GAIA, uses 6-7x fewer input tokens than a ReAct agent, and adds 8.0 points to a proprietary agent on BrowseComp.
This preprint proposes a unified post-training quantization framework for large language models. It derives a Hessian-guided approximation of the output change caused by joint weight and activation quantization, using a tractable Fisher approximation. Quantization ranges, linear transformations and weight rounding are optimized together under one output-sensitivity objective. The authors report gains over state-of-the-art transformation and rounding methods in channel-wise and block-wise settings.
Decompose, Estimate, Select is a coverage-aware retrieval pipeline for long-form RAG. It splits a query into subquestions, ranks candidates with a coverage-trained bi-encoder, predicts a document-by-subquestion coverage matrix with a cross-encoder, and picks documents by greedy maximum-marginal coverage. On TREC RAG 2024, Ξ±-nDCG@10 rises from 0.625 to 0.702 (gold-nugget bound 0.718). A grounded self-feedback loop raises V_strict by 0.086 over two passes.
RAGbi-encodercross-encoder+1
TREC RAG 2024Ξ±-nDCG@10V_strict
llm-apps, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
MAServeBench is a benchmark that treats multi-agent collaboration as a configurable dimension of LLM serving workloads. Its composable stages model how agents coordinate and exchange context in documented multi-agent workflows, and Hierarchical Semantic Traces record operations and dependencies for reproducible comparisons. Task-linked measurements tie workloads to GPU/NPU behavior, showing serving efficiency does not always mean timely task completion. Code is released.
LLM
MAServeBencharXiv
evals, inference, agentsΒ· Added 31 Oct 2026, published 29 Oct 2026
Nadim Kobeissi's ePrint paper 2026/2683 argues that ProVerif and Tamarin should be called symbolic protocol analyzers, not provers. It audits their published soundness arguments, finding false statements, missing proofs and gaps between paper and implementation, and reproduces reported Tamarin bugs that accept false claims. It also covers model gaps in parsers and state machines, weak secrecy queries and unjustified primitive equations. The page is an abstract only.
ProVerifTamarin
Nadim Kobeissicomputational soundnesssymbolic model
M-PFT is an angular-margin method for prompt tuning of vision-language models that fixes base-to-novel dual miscalibration (underconfidence on base classes, overconfidence on novel classes). It adds two losses: minimizing the maximum cosine similarity between tunable class embeddings, and minimizing angular distance to each zero-shot embedding. Tests on 9 prompt-tuning methods and 15 datasets show lower novel-class calibration error at comparable accuracy and negligible overhead.
vision-language modelprompt tuning
arXiv
research, multimodal, trainingΒ· Added 31 Oct 2026, published 29 Oct 2026
Cryptanalysis of VTBSAE, a verifier-based PAKE protocol for the Industrial Internet of Things, showing it fails to resist client impersonation after server compromise. The authors propose E-VTBSAEv1, which binds the client point E1 to fresh server randomness, and prove AKE security and forward secrecy in the random-oracle model under CDH. Raspberry Pi 4B tests show runtime comparable to VTBSAE with the same communication and storage overhead.
Raspberry Pi 4BPAKE
VTBSAEE-VTBSAEv1client impersonation after server compromise+3
Identity & Access, Hardware & RFΒ· Added 31 Oct 2026, published 29 Oct 2026
MedCBR is a two-stage alignment framework for medical vision language models. Difficulty Weighted Boundary Preference Optimization (DBPO) trains on counterfactual preference pairs at the model's capability frontier. Entropy Gated Grounded Policy Optimization (EGPO) then applies distilled clinical rubrics according to prediction entropy. It reaches 61.2% average accuracy on six medical benchmarks, with the largest gains on differential diagnosis, and the two stages combine super-additively.
SEAL is a defense against indirect prompt injection for local LLM agents. It learns a soft prompt and scoring readout offline, then uses a frozen task model to score and delete injected tokens from tool outputs, with queries built from trusted initial context. On AgentDojo, attack success falls from 34.6% to 0.1% and Joint rises from 51.4% to 77.5%, using about 127x fewer defense tokens than PISanitizer.
LLMAgentDojoAgentDyn+1
indirect prompt injection
AI SecurityΒ· Added 31 Oct 2026, published 29 Oct 2026
HO-ASPA (Higher-Order Adjusted Spearman Coefficient Analysis) is a rank-based, non-profiling side-channel distinguisher. The paper derives its optimal form against two-share Boolean masking and inner product masking under Hamming weight and LSB leakage. It finds no significant gap between higher and lower orders for HO-ASPA, unlike HO-CPA, and reports that it beats Kruskal-Wallis-based attacks in simulated and real-world experiments.
HO-ASPAHO-CPAKruskal-Wallis+2
Hardware & RFΒ· Added 31 Oct 2026, published 29 Oct 2026
High Precision Noise Cleaning for CKKS is an IACR ePrint preprint (2026/2675) by Zhihao Li, Cheng Hong and Tao Wei. It composes lower-degree Hermite polynomials to reach output noise bounds such as 2^-300 for noise flooding, and uses this to build an asynchronous threshold CKKS scheme with strong simulation security in the random oracle model. In OpenFHE, binary-message cleaning falls from 9.63 s to 1.54 s.
REALM (Required Execution Assets Linked to a Mission) is an approach to applying least privilege to LLM agents by identifying the assets a task needs, including ones discovered at runtime. It models assets as Dependency Graph nodes, separates discovering an asset from authorizing it, and exposes resource-bound Partial APIs inside a Task Container. On AgentDyn with qwen3.5-flash, attack success rate falls from 16.6% to 0.2%, with 40.0% task completion on clean tasks and 36.4% on injected cases.
QwenLLMAgentDyn
prompt injectionleast privilegeREALM
AI SecurityΒ· Added 31 Oct 2026, published 29 Oct 2026
DICE is a blockchain-powered federated learning (BCFL) algorithm for customizing large pretrained models among untrusted Web3 participants. It combines dynamic progressive learning and model slice verification, so hardware-limited participants can train part of the model, with a tree-structured committee consensus protocol and an efficiency-driven incentive scheme to scale to more data holders. Experiments cover vision and language models.
PrivacyEvalClaw is a tool-augmented deliberative agent that audits privacy leakage in LLM jailbreak responses. The paper is by Krishna Kanth Nakka and co-authors, and it was accepted to AACL-IJCNLP 2026 (arXiv 2610.36711, cs.CR). The tweet gives only the title, authors and venue, so the method and results are not described.
LLM
PrivacyEvalClaw
AI SecurityΒ· Added 30 Oct 2026, published 29 Oct 2026
PrivacyEvalClaw is a tool-augmented deliberative agent for auditing privacy leakage in LLM responses produced under jailbreak prompts. By Krishna Kanth Nakka and five co-authors, the paper is on arXiv (cs.CR) and accepted to AACL-IJCNLP 2026. The fetched post gives only the title, authors and venue, so method details and results are not available here.
Paper studying whether self-evolution weakens personal agents' robustness to indirect prompt injection (IPI). A two-phase method has the user ask the agent to follow tasks in external content, then tests with a third-party injected task the user did not authorize. Against the non-evolving base agent, attack success rose 14.0 to 48.8 points under skill evolution and 3.0 to 39.3 points under reinforcement learning. Code is released.
indirect prompt injection
AI SecurityΒ· Added 31 Oct 2026, published 29 Oct 2026
Proof-Carrying Messages (PCM) is a protocol where each claim an AI agent sends carries evidence: records from an append-only Merkle log the agent cannot write, with inclusion proofs, signed by an attesting signer. Receivers verify offline in about 1 ms and label each claim proven, unproven or failed. On 926 messages (712 lies in 16 classes), PCM accepts 10 lies (1.4%), all stale evidence; plain A2A accepts all 712.
A2AModel Context ProtocoluAgents+5
arXiv
agents, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
Proof-Carrying Messages (PCM) is a protocol where each claim an AI agent sends carries evidence: records from an append-only Merkle log the agent cannot write, with inclusion proofs, signed by an attesting signer it cannot control. Receivers verify offline and mark each claim proven, unproven or failed. On 926 messages (712 lies in 16 classes) PCM accepted 10 lies, all stale evidence (1.4%), while plain A2A accepted all 712. Verification takes about 1 ms.
Model Context ProtocolA2AuAgents
prompt injectionRFC 9421SCITT+3
AI SecurityΒ· Added 31 Oct 2026, published 29 Oct 2026
TrussTutor-Bench is a benchmark of 200 verified two-dimensional truss problems from statics, paired with automatically generated student solutions. It evaluates thirteen LLMs on solution accuracy (64.5% to 97.0%), error detection (90.4% to 100%) and sycophancy. Models rejected their own correct solutions in 1.6% to 50.0% of cases when a student insisted the work was wrong. Sycophancy separated models; error detection did not.
LLM
TrussTutor-BencharXiv
evals, researchΒ· Added 31 Oct 2026, published 28 Oct 2026
Gold vs. Translation is an October 2026 arXiv study of whether DeepL-translated MMLU measures the same thing as native benchmarks. It compares seven pairings against six native benchmarks in five Nordic and Baltic languages, using 29 models. Translated MMLU adds predictive value for curriculum-overlap benchmarks such as Skolprov (ΞRΒ²=.128) but not for Sweden-specific SwedishFacts (p=.242).
LLM
MMLUDeepLSkolprov+2
evals, researchΒ· Added 30 Oct 2026, published 28 Oct 2026
huggingface.copaperπ§ AI
