The paper treats a benchmark as an empirical measure over item difficulty and formalizes two failure modes: measure mismatch (benchmark difficulty distribution differs from the intended one) and coverage gaps (uncovered difficulty regions that hide localized training progress). It proposes two diagnostic metrics with theoretical guarantees and analyzes 25 popular benchmarks across 7 domains using estimated item difficulties. It also introduces BenchFill, a difficulty-targeted item rewriting algorithm that reduces local coverage gaps.
WorkFactory is a framework that builds coherent, executable professional tasks for training working agents. It plans an information support graph from professional intents, then uses probe executions to find and repair missing or inconsistent task information. The authors built 112,686 tasks across 612 occupations and fine-tuned Qwen models at 4B, 27B and 35B-A3B (WorkPilot). WorkPilot-35B-A3B rose from 879 to 1565 Elo on GDPval-AA v2.1.
GPT-Rosalind-6 System Card Addendum is OpenAI's safety documentation for a life-sciences model that shares weights with GPT-6 Astra. It rates the model High in Biological and Chemical and Critical in Cybersecurity under the Preparedness Framework. It reports biology evaluations, trusted-access controls, biosecurity refusal pass rates, a historical cyber jailbreak regression (14.3% task success), and sabotage and monitor-evasion results.
GPT-Rosalind-6 System Card Addendum describes OpenAI's life-sciences reasoning model, which uses the same weights as GPT-6 Astra and is offered only through trusted access. It reports life-science benchmark gains over GPT-Rosalind-5.5 (MedChemBench v7 49.6% vs 28.9%), High-capability Biological and Chemical Preparedness results, and below-threshold Critical bio design evaluations. It also covers access controls, refusal evaluations, cyber jailbreak regression and alignment tables.
GPT-6 AstraGPT-Rosalind-6GPT-Rosalind-5.5
OpenAIPreparedness FrameworkMedChemBench v7+16
ai-safety, evals, modelsΒ· Published and added 30 Oct 2026
world2vec is a method for learning mechanistic world models from observations and actions alone, without reward information. Its latent dynamics reproduce the true system over long prediction horizons using at least one order of magnitude fewer parameters than DreamerV3, TD-MPC2 and LeWM. The authors report state-of-the-art control of continuous environments from pixels and accurate system identification across a benchmark of 71 attractors.
world modellatent dynamics learning
world2vecDreamerV3TD-MPC2+2
researchΒ· Added 31 Oct 2026, published 29 Oct 2026
TextSeal2 is a distortion-free LLM text watermark that merges two or more keys into one virtual key and couples it to fresh noise through AutoMark's Gaussian copula. Under speculative decoding it verifies keyed drafts with an extra exponential race, so output matches the target model and acceptance rate is unchanged. On three LLMs at a 10^-3 false-positive rate, it detects 8 to 22 points more 100-token answers than TextSeal.
VeriCUA is a platform-agnostic framework that verifies task completion by computer-using agents through agentic exploration. The verifier inspects context to build a task rubric, then picks GUI or code checks for unresolved requirements and combines findings under explicit evidence rules. It works online and offline across desktop, mobile and web, and agrees better with reference labels than the baselines, with the largest offline gains on hard cases.
vision-language modelVeriCUA
arXiv
agents, evals, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
Multi-View MoE is a sparse mixture-of-experts design that uses several independently initialized routers to select experts, then reuses their relative scores to mix learned latent views into each selected expert's input. Each expert still runs one forward pass, so active-expert count, FLOPs and all-to-all volume are unchanged. On a 24-layer 10B-A1B model trained on 708B tokens, it gains 1.18 points on a nine-task mean; Multi-Router alone gains 0.6.
Mixture of ExpertsTransformerMulti-View MoE+1
arXiv
researchΒ· Added 31 Oct 2026, published 29 Oct 2026
This paper derives bounds on predicting an LLM's decode length from prefill-phase hidden states, a primitive for inference scheduling. It models decoding as a stopped stochastic process, giving per-prompt normality and an affine variance law Ο=Ξ±ΞΌ+b. Predictors (MLP, EGTP, FiLM, soft-label, Mixture-of-Experts) converge near the bound, so system error dominates model error. It proposes fixing model, data and sampling in benchmarks and reporting relative-error metrics.
LLMMixture of ExpertsMLP+2
arXiv
research, inference, evalsΒ· Added 31 Oct 2026, published 29 Oct 2026
PhysCount is a benchmark of 546 real-world images and 4,595 human-written questions in 651 target-specific groups. It tests whether multimodal LLMs can separate visible appearances (direct view, reflections, glass) from the physical entities behind them. Across 40 MLLMs, counting appearances is much easier than linking them to entities, and the best model answers every question correctly in only 13.36% of groups.
vision-language model
PhysCountarXiv
evals, multimodal, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
Glance is a RAG method in which the generator also reranks. It adds a lightweight decision adapter and pointer head that score candidate passages from one shared prefill, so a single resident 9B backbone does generation and reranking. After 39 minutes of MS MARCO fine-tuning it reaches nDCG@10 of 0.741 on SciFact and 0.345 on NFCorpus, against 0.721 and 0.326 for bge-reranker-v2-m3. On 300 HotpotQA questions it improves answer F1 by 0.045 over cross-encoder-selected context. Evaluation covers two retrieval datasets, one QA dataset and one backbone family.
RAGLLMbge-reranker-v2-m3
MS MARCOSciFactNFCorpus+1
llm-apps, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
On-policy state training is a method for training chain embedding retrievers for multi-hop RAG on the states the encoder itself reaches, including those built on its own wrong retrievals. Each state is labeled with the earliest missing gold passage, other missing gold passages are masked, and updates are contrastive and mixed with teacher-forced states. It retrieves complete evidence more often than teacher-forced training and graph-based systems, and transfers to harder long-chain HotpotQA and 2WikiMultiHopQA variants the authors built.
RAGon-policy state trainingcontrastive learning+1
HotpotQA2WikiMultiHopQAarXiv
research, llm-apps, trainingΒ· Added 31 Oct 2026, published 29 Oct 2026
CEDAR (Clinical Evidence-Driven Agent for Risk-prediction) is an evidence-citing, self-auditing LLM pipeline for predicting pediatric diabetes outcomes at 24 months from 18 months of records. On 2,489 youth with type 2 diabetes and 4,018 with type 1, pooled AUROC was 0.815 versus 0.656 to 0.700 for trained ML in T2D, and 0.756 versus 0.603 to 0.631 in T1D. Only 4 of 70,246 cited values mismatched the chart. Two pediatric endocrinologists matched its predictions in about nine of ten cases.
LLM
CEDARarXiv
research, agentsΒ· Added 31 Oct 2026, published 29 Oct 2026
EpiCoach is an agentic system that treats one cross-country mountain biker as an N-of-1 experiment. Claims live in an evidence-gated hypothesis registry, and each must make a probabilistic prediction before a session. Predictions come from a digital twin fitted to heart rate, GPS and altitude. In a replay over eighteen months the twin beat standard forecasters on heart rate but not segment time, and the agent loop matched a static model. In the first three live weeks, the ledger sealed 20 predictions and scored 14, and a safety officer vetoed the agent twice. The registry, benchmark and protocol are released.
LLMBayesian experimental designdigital twin+1
EpiCoacharXiv
agents, research, evalsΒ· Added 31 Oct 2026, published 29 Oct 2026
Decompose, Estimate, Select is a coverage-aware retrieval pipeline for long-form RAG. It splits a query into subquestions, ranks candidates with a coverage-trained bi-encoder, predicts a document-by-subquestion coverage matrix with a cross-encoder, and picks documents by greedy maximum-marginal coverage. On TREC RAG 2024, Ξ±-nDCG@10 rises from 0.625 to 0.702 (gold-nugget bound 0.718). A grounded self-feedback loop raises V_strict by 0.086 over two passes.
RAGbi-encodercross-encoder+1
TREC RAG 2024Ξ±-nDCG@10V_strict
llm-apps, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
MAServeBench is a benchmark that treats multi-agent collaboration as a configurable dimension of LLM serving workloads. Its composable stages model how agents coordinate and exchange context in documented multi-agent workflows, and Hierarchical Semantic Traces record operations and dependencies for reproducible comparisons. Task-linked measurements tie workloads to GPU/NPU behavior, showing serving efficiency does not always mean timely task completion. Code is released.
LLM
MAServeBencharXiv
evals, inference, agentsΒ· Added 31 Oct 2026, published 29 Oct 2026
This paper measures the energy cost of speculative decoding in SGLang with CUDA graphs on H100 and H200 GPUs, at batch sizes up to 64 across 5 corpora. Savings depend on drafted-token acceptance, which is lower on conversation than code and lower on Romansh than English. Savings shrink as batch size grows, and block drafters can push verification into the compute-bound regime, where speculation raises energy per token.
SGLangCUDALLM+4
NVIDIAarXivRomansh
inference, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
A five-task benchmark tests multimodal LLM color judgment against CIE colorimetric ground truth: AWB pair judgment, skin-tone bias direction, CCT estimation, Delta E 2000 pair ordering, and color naming under changing illuminants. Eight endpoints were tested. On ColorChecker-24 stimuli, closed frontier models reach CCT MAE of 715-1056 K versus 1358-2133 K for open-weight models, and a non-ColorChecker control shows this gap is partly template-dependent.
vision-language modelQwen3-VLGLM+2
GLM-4.5VGPT-5.4Claude-Opus-4.7+4
evals, multimodal, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
Kolesnikov and Novoselov give block basis reduction (BKZ) algorithms for linear codes that need polynomial or quasi-polynomial numbers of calls to a minimum weight codeword oracle at fixed block size. The earlier Ghentiyala and Stephens-Davidowitz bounds are exponential. The paper adds a truncated BKZ and a DeepBKZ variant, with an implementation for block sizes 2 to 24 and approximation-factor estimates on random codes.
AutoBench-CT is a modular benchmark for constant-time analysis tools, with manually verified per-violation ground truth across 64 cryptographic tasks. It scores seven tools on operational performance, detection quality and usability. The authors explain why CT-grind (2010) is still the most used tool, list recurring limitations across tools, and propose a composite pipeline for stronger constant-time guarantees. Accepted at ACSAC 2026.
AutoBench-CTCT-grind
ACSAC
Hardware & RF, AppSecΒ· Added 31 Oct 2026, published 29 Oct 2026
M-PFT is an angular-margin method for prompt tuning of vision-language models that fixes base-to-novel dual miscalibration (underconfidence on base classes, overconfidence on novel classes). It adds two losses: minimizing the maximum cosine similarity between tunable class embeddings, and minimizing angular distance to each zero-shot embedding. Tests on 9 prompt-tuning methods and 15 datasets show lower novel-class calibration error at comparable accuracy and negligible overhead.
vision-language modelprompt tuning
arXiv
research, multimodal, trainingΒ· Added 31 Oct 2026, published 29 Oct 2026
Cryptanalysis of VTBSAE, a verifier-based PAKE protocol for the Industrial Internet of Things, showing it fails to resist client impersonation after server compromise. The authors propose E-VTBSAEv1, which binds the client point E1 to fresh server randomness, and prove AKE security and forward secrecy in the random-oracle model under CDH. Raspberry Pi 4B tests show runtime comparable to VTBSAE with the same communication and storage overhead.
Raspberry Pi 4BPAKE
VTBSAEE-VTBSAEv1client impersonation after server compromise+3
Identity & Access, Hardware & RFΒ· Added 31 Oct 2026, published 29 Oct 2026
MedCBR is a two-stage alignment framework for medical vision language models. Difficulty Weighted Boundary Preference Optimization (DBPO) trains on counterfactual preference pairs at the model's capability frontier. Entropy Gated Grounded Policy Optimization (EGPO) then applies distilled clinical rubrics according to prediction entropy. It reaches 61.2% average accuracy on six medical benchmarks, with the largest gains on differential diagnosis, and the two stages combine super-additively.
Paper by Yi Ouyang, Wei Wei and Zheng Xu on the minimum degree of an isogeny from a supersingular elliptic curve E to its Frobenius conjugate E^(p). It bounds the probability that this degree is smooth, which makes Wesolowski's p^(1/3+o(1)) attack on the supersingular endomorphism ring problem unconditional. It also gives a uniform asymptotic formula, with Dirichlet L-values in the main term, for the probability that a connection isogeny of degree n exists.
supersingular endomorphism ring problemWesolowskiYi Ouyang+2
HO-ASPA (Higher-Order Adjusted Spearman Coefficient Analysis) is a rank-based, non-profiling side-channel distinguisher. The paper derives its optimal form against two-share Boolean masking and inner product masking under Hamming weight and LSB leakage. It finds no significant gap between higher and lower orders for HO-ASPA, unlike HO-CPA, and reports that it beats Kruskal-Wallis-based attacks in simulated and real-world experiments.
HO-ASPAHO-CPAKruskal-Wallis+2
Hardware & RFΒ· Added 31 Oct 2026, published 29 Oct 2026
RetroChessBench is a 12-task benchmark for retrograde chess: reconstructing legal game histories, exact-length constructions, unique endings and exhaustive answer sets, with references, rubrics and a terminal-reward RL environment. GPT 6 Astra High fails to give the required final answer in 22 of 24 attempts without tools and 18 of 24 with code execution. Mean terminal RL rewards are 14.75% and 27.19%. The authors link the failures to scarce worked derivations in web training data.
GPT-6 AstraLLM
RetroChessBench
evals, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
High Precision Noise Cleaning for CKKS is an IACR ePrint preprint (2026/2675) by Zhihao Li, Cheng Hong and Tao Wei. It composes lower-degree Hermite polynomials to reach output noise bounds such as 2^-300 for noise flooding, and uses this to build an asynchronous threshold CKKS scheme with strong simulation security in the random oracle model. In OpenFHE, binary-message cleaning falls from 9.63 s to 1.54 s.
This arXiv preprint (October 2026) tests a sparse autoencoder as a factor model for daily cryptocurrency returns, with equities as a secondary test. SAE portfolios show consistently lower volatility than common benchmarks, linked to lower correlations among selected assets. They do not beat a simple contrarian strategy on risk-adjusted return, and factor identities change between training runs.
sparse autoencoder
arXiv
researchΒ· Added 31 Oct 2026, published 29 Oct 2026
REALM (Required Execution Assets Linked to a Mission) is an approach to applying least privilege to LLM agents by identifying the assets a task needs, including ones discovered at runtime. It models assets as Dependency Graph nodes, separates discovering an asset from authorizing it, and exposes resource-bound Partial APIs inside a Task Container. On AgentDyn with qwen3.5-flash, attack success rate falls from 16.6% to 0.2%, with 40.0% task completion on clean tasks and 36.4% on injected cases.
QwenLLMAgentDyn
prompt injectionleast privilegeREALM
AI SecurityΒ· Added 31 Oct 2026, published 29 Oct 2026
DICE is a blockchain-powered federated learning (BCFL) algorithm for customizing large pretrained models among untrusted Web3 participants. It combines dynamic progressive learning and model slice verification, so hardware-limited participants can train part of the model, with a tree-structured committee consensus protocol and an efficiency-driven incentive scheme to scale to more data holders. Experiments cover vision and language models.
huggingface.copaperπ§ AI
