The paper treats a benchmark as an empirical measure over item difficulty and formalizes two failure modes: measure mismatch (benchmark difficulty distribution differs from the intended one) and coverage gaps (uncovered difficulty regions that hide localized training progress). It proposes two diagnostic metrics with theoretical guarantees and analyzes 25 popular benchmarks across 7 domains using estimated item difficulties. It also introduces BenchFill, a difficulty-targeted item rewriting algorithm that reduces local coverage gaps.
TextSeal2 is a distortion-free LLM text watermark that merges two or more keys into one virtual key and couples it to fresh noise through AutoMark's Gaussian copula. Under speculative decoding it verifies keyed drafts with an extra exponential race, so output matches the target model and acceptance rate is unchanged. On three LLMs at a 10^-3 false-positive rate, it detects 8 to 22 points more 100-token answers than TextSeal.
TextSeal2 is a distortion-free watermark for LLM output that keeps its signal under speculative decoding. It merges two or more keys into one virtual key coupled to fresh noise through AutoMark's Gaussian copula, and verifies drafts with an extra exponential race. On three LLMs it detects 8 to 22 points more 100-token answers than TextSeal at a 10^-3 false-positive rate.
LLMspeculative decodingGumbel-max watermarking+1
TextSeal2TextSealAutoMark+2
research, inferenceΒ· Added 31 Oct 2026, published 29 Oct 2026
This paper derives bounds on predicting an LLM's decode length from prefill-phase hidden states, a primitive for inference scheduling. It models decoding as a stopped stochastic process, giving per-prompt normality and an affine variance law Ο=Ξ±ΞΌ+b. Predictors (MLP, EGTP, FiLM, soft-label, Mixture-of-Experts) converge near the bound, so system error dominates model error. It proposes fixing model, data and sampling in benchmarks and reporting relative-error metrics.
LLMMixture of ExpertsMLP+2
arXiv
research, inference, evalsΒ· Added 31 Oct 2026, published 29 Oct 2026
This preprint proposes a unified post-training quantization framework for large language models. It derives a Hessian-guided approximation of the output change caused by joint weight and activation quantization, using a tractable Fisher approximation. Quantization ranges, linear transformations and weight rounding are optimized together under one output-sensitivity objective. The authors report gains over state-of-the-art transformation and rounding methods in channel-wise and block-wise settings.
Decompose, Estimate, Select is a coverage-aware retrieval pipeline for long-form RAG. It splits a query into subquestions, ranks candidates with a coverage-trained bi-encoder, predicts a document-by-subquestion coverage matrix with a cross-encoder, and picks documents by greedy maximum-marginal coverage. On TREC RAG 2024, Ξ±-nDCG@10 rises from 0.625 to 0.702 (gold-nugget bound 0.718). A grounded self-feedback loop raises V_strict by 0.086 over two passes.
RAGbi-encodercross-encoder+1
TREC RAG 2024Ξ±-nDCG@10V_strict
llm-apps, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
MAServeBench is a benchmark that treats multi-agent collaboration as a configurable dimension of LLM serving workloads. Its composable stages model how agents coordinate and exchange context in documented multi-agent workflows, and Hierarchical Semantic Traces record operations and dependencies for reproducible comparisons. Task-linked measurements tie workloads to GPU/NPU behavior, showing serving efficiency does not always mean timely task completion. Code is released.
LLM
MAServeBencharXiv
evals, inference, agentsΒ· Added 31 Oct 2026, published 29 Oct 2026
AutoBench-CT is a modular benchmark for constant-time analysis tools, with manually verified per-violation ground truth across 64 cryptographic tasks. It scores seven tools on operational performance, detection quality and usability. The authors explain why CT-grind (2010) is still the most used tool, list recurring limitations across tools, and propose a composite pipeline for stronger constant-time guarantees. Accepted at ACSAC 2026.
AutoBench-CTCT-grind
ACSAC
Hardware & RF, AppSecΒ· Added 31 Oct 2026, published 29 Oct 2026
Turing LLM post-training combines calibrated multi-teacher on-policy distillation (C-MOPD) with an execution path for dynamically routed mixture-of-experts models. C-MOPD matches each teacher on a local Top-K support with teacher-wise loss normalization and clipping. On Turing-20B-A2B it scores 0.9988 on four-domain integration, beats Qwen3-8B on math and code (85.92 vs 80.33 mean), and cuts consolidation step time from 398.7 to 139.3 seconds in a PPU profile.
Mixture of ExpertsC-MOPDon-policy distillation+1
Turing-20B-A2BarXiv
training, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
Cryptanalysis of VTBSAE, a verifier-based PAKE protocol for the Industrial Internet of Things, showing it fails to resist client impersonation after server compromise. The authors propose E-VTBSAEv1, which binds the client point E1 to fresh server randomness, and prove AKE security and forward secrecy in the random-oracle model under CDH. Raspberry Pi 4B tests show runtime comparable to VTBSAE with the same communication and storage overhead.
Raspberry Pi 4BPAKE
VTBSAEE-VTBSAEv1client impersonation after server compromise+3
Identity & Access, Hardware & RFΒ· Added 31 Oct 2026, published 29 Oct 2026
High Precision Noise Cleaning for CKKS is an IACR ePrint preprint (2026/2675) by Zhihao Li, Cheng Hong and Tao Wei. It composes lower-degree Hermite polynomials to reach output noise bounds such as 2^-300 for noise flooding, and uses this to build an asynchronous threshold CKKS scheme with strong simulation security in the random oracle model. In OpenFHE, binary-message cleaning falls from 9.63 s to 1.54 s.
REALM (Required Execution Assets Linked to a Mission) is an approach to applying least privilege to LLM agents by identifying the assets a task needs, including ones discovered at runtime. It models assets as Dependency Graph nodes, separates discovering an asset from authorizing it, and exposes resource-bound Partial APIs inside a Task Container. On AgentDyn with qwen3.5-flash, attack success rate falls from 16.6% to 0.2%, with 40.0% task completion on clean tasks and 36.4% on injected cases.
QwenLLMAgentDyn
prompt injectionleast privilegeREALM
AI SecurityΒ· Added 31 Oct 2026, published 29 Oct 2026
PrivacyEvalClaw is a tool-augmented deliberative agent for auditing privacy leakage in LLM responses produced under jailbreak prompts. By Krishna Kanth Nakka and five co-authors, the paper is on arXiv (cs.CR) and accepted to AACL-IJCNLP 2026. The fetched post gives only the title, authors and venue, so method details and results are not available here.
This paper studies whether vision-language models can infer how long a person stays somewhere (living, staying, or leaving) from geotagged social media posts. The authors build a new corpus of geotagged tweets co-annotated by humans and LLMs. State-of-the-art models struggle even with multi-turn Chain-of-Thought prompting, often missing contextual cues and subtle hashtags while over-relying on prominent visual details.
vision-language modelLLM
arXiv
research, evals, multimodalΒ· Added 30 Oct 2026, published 28 Oct 2026
AskStream is a multi-turn benchmark for streaming video understanding that adds a third proactive action, ask, alongside wait and answer, for cases where video or user information is incomplete. It has 438 videos, 2,714 manually annotated frame-level time windows (wait, ask, answer), four task categories and eight subtasks. The strongest model reaches only 52.18% on the proposed Ask-F1 metric, in both multiple-choice and open-ended settings.
multimodal large language model
AskStreamAsk-F1arXiv
evals, multimodal, researchΒ· Added 30 Oct 2026, published 28 Oct 2026
QbD-VLA is a mixed-precision post-training quantization method for vision-language-action models. It uses task success from short closed-loop evaluations to assign weight and activation bit widths, accounting for interactions between components. On four VLA models across all four LIBERO suites it holds task performance at about 4-bit weights and under 6-bit activations. The gain on SIMPLER is small. On a Franka Research 3 arm, X-VLA shows no detected loss, but Ο0.5 fails.
vision-language-action model
QbD-VLALIBEROSIMPLER+4
research, inference, multimodalΒ· Added 30 Oct 2026, published 28 Oct 2026
Secure Transaction Verification and Authorization (STVA) is a draft Standards Track ERC (created October 2026) by @MATOBOYCRYPTO65. It proposes structured human-readable metadata and semantic intent for transactions and UserOperations, an independent simulation and verification interface, and scoped, revocable capability tokens for privileged operations. It targets blind signing and frontend supply-chain attacks, citing the February 2025 Bybit Safe multisig theft. The page gives only the abstract, motivation and core concepts.
Beyond the Event Horizon of Keccak extends a 2011 observation that a path in the Keccak sponge from a zero-capacity state back to one yields infinite second-preimage families. Three theorems cover loops from any capacity value x, k-block loops and cycles, and Joux-style multicollisions with search cost independent of k. Verified examples run on Keccak-f[200] and Keccak-f[1600] with c=32; it also proposes bridge-based protocols.
This paper measures job-based doubt in LLM fact-checkers: judging a claim false more often when the poster has a lower-status job. It compares base and post-trained checkpoints of six model families on fact-checked claims with randomised poster profiles. Only the open TΓΌlu-3 recipe (OLMo-2 32B, TΓΌlu-3 70B) creates the effect, under one percentage point but reliable. In OLMo-2 32B the flip comes from one residual-stream direction and appears at supervised fine-tuning, confirmed on 400 new claims. None of seven commercial models doubts lower-status posters more.
LLMTransformer
OLMo-2 32BTΓΌlu-3TΓΌlu-3 70B+3
research, ai-safety, trainingΒ· Added 30 Oct 2026, published 28 Oct 2026
This paper measures how much a model's final answer depends on its own written chain of thought, using a synthetic box-swap task with exact ground truth. It edits one step in the model's chain to a wrong but valid state and checks whether the answer follows. Across 25 open-weight models, reliance rises with size and saturates (Qwen3 from 16-18% at 0.6B to 97-100% at 14B). Reliance tracks capability (r = 0.99) and is unchanged by post-training.
Qwen3GemmaLlama+2
Gemma 3Llama 3arXiv
research, ai-safety, evalsΒ· Added 30 Oct 2026, published 28 Oct 2026
LLMON (LLM Object Notation) is a framework that separates instructions from data and requires explicit identifier-based execution. The authors post-train Granite-4.1 3B and 8B base models with SFT then RL on a 4.6B-token structured corpus covering general, RAG and tool-calling data, with distractor variants. Attack success rates on indirect prompt-injection benchmarks fall from 53-100% to 0.02-5.6%, and general capability is preserved. Attention-based metrics diagnose failures per transformer layer.
LLMONGranite 4.1LLM
prompt injectionindirect prompt injection
AI SecurityΒ· Added 30 Oct 2026, published 28 Oct 2026
LLMON (LLM Object Notation) is a framework that separates executable instructions from passive data and requires explicit identifier-based execution. The paper post-trains Granite-4.1 3B and 8B base models on a 4.6B-token corpus of general, RAG and tool-calling data with distractor variants, using SFT then RL. Attack success rates on indirect prompt-injection benchmarks fall from 53-100% to 0.02-5.6%, with general capabilities preserved. It also adds attention-based per-layer diagnostics.
LLMRAGTransformer
LLMONGranite-4.1arXiv
training, researchΒ· Added 30 Oct 2026, published 28 Oct 2026
This paper proposes an fMLLR-based compensation mechanism that cuts the fine-tuning data needed in transfer learning for speech emotion recognition under train-test acoustic mismatch. An autoencoder reduces trainable parameters, and an affine transform in the encoder latent space models rotations and shifts. In a human-robot interaction task, mean CCC rose 16.8% with 12.9% of the training set and 14.4% with 50%.
fMLLRautoencoder
arXiv
research, multimodal, trainingΒ· Added 30 Oct 2026, published 27 Oct 2026
The paper proposes POB-ICM (contingency matrix with Chi-square independence testing) and POB-UNBP (a uniformity null model under permutation-based evaluation) to detect positional bias in top-1, single-turn LLM recommenders. On a RAG product-recommendation testbed with 2000 queries and 20 permutations each (40,000 instances), both methods reject positional independence in all evaluated models. Three metrics quantify the effect, and the bias persists across candidate-set size and temperature. Code is public.
LLMRAG
arXivPOB-ICMPOB-UNBP
research, evals, llm-appsΒ· Added 30 Oct 2026, published 27 Oct 2026
Locating Sycophancy Across Training Stages in Open Language Models measures sycophancy in 13 open checkpoints from 5 lineages, from base model through supervised fine-tuning and preference optimization to release. It finds two behaviors with different origins: deference to a claim in the prompt exists in every base model, while capitulation under user objection is largely absent from base models and present in every post-trained checkpoint. Post-trained models often drop firmly held answers as readily as loosely held ones when the objection names an alternative.
LLM
arXiv
ai-safety, research, evalsΒ· Added 30 Oct 2026, published 27 Oct 2026
Know, then Draw is a learning paradigm for visual generation that uses autoregressive pretraining to acquire semantic knowledge and diffusion post-training for high-fidelity rendering. Controlled comparisons find autoregression the stronger pretraining objective, and semantic quality matters more than AR-only generation quality. This motivates a purely semantic tokenizer trained with language supervision only. A 3B text-to-image model trained in 6,600 H100 GPU-hours scores 86 on GenEval and 87 on DPG-Bench.
diffusion modelTransformer
arXivGenEvalDPG-Bench+1
research, multimodal, trainingΒ· Added 28 Oct 2026, published 27 Oct 2026
This paper presents an annotated egocentric tennis dataset (egocentric video synced with five allocentric views, global positions, contact times, stroke types) and evaluates image encoders, video encoders and vision-language models on seven prediction tasks. Best models reach 0.14 m player and 0.54 m ball localization error and 0.14 s contact-time error. Depth input helps 3D ball localization, and longer histories do not help forecasting. A conditional diffusion model generates whole-body tennis motion in real time.
vision-language modeldiffusion modelTransformer
arXiv
research, multimodal, evalsΒ· Added 28 Oct 2026, published 27 Oct 2026
This paper defines a character naming task for films: given a name spoken in dialogue and its video context, the model must pick which visible character bears that name. It provides a dataset with annotations for character naming and first appearances, a video-and-dialogue model, and a comparison with proprietary multimodal LLMs. It also proposes a progressive framework that builds a character bank as the film plays, without cast lists or actor photographs.
multimodal large language model
arXiv
research, multimodalΒ· Added 28 Oct 2026, published 27 Oct 2026
FeatureArc is a benchmark for coding agents on evolving features, built by a pipeline that mines repository histories into multi-stage tasks across four programming languages. It has 73 manually reviewed tasks of 2 to 10 stages, validated with fail-to-pass and pass-to-pass tests. Across 10 agents, pooled success drops from 62.7% at the first stage to 28.5% for the whole feature. Inherited defects appear in 72.9% of failed later stages.
LLM
FeatureArcTencentHunyuan+1
evals, coding-agents, researchΒ· Added 28 Oct 2026, published 27 Oct 2026
This paper studies how multimodal LLMs refuse questions that an image cannot answer, using the MoHoBench benchmark on three MLLMs. It varies the meaning and placement of refusal instructions, compares supervised and preference-based post-training, and intervenes on a refusal-related activation direction. Instructions change refusal at the sample level in ways aggregate rates hide. Post-training raises refusal most but can mix refusal with attempted answers. Activation intervention gives smaller, transferable changes.
vision-language modelLLM
MoHoBencharXiv
ai-safety, multimodal, researchΒ· Added 28 Oct 2026, published 27 Oct 2026
huggingface.copaperπ§ AI
