This paper replicates Chen et al.'s study of LLM-based smart contract vulnerability detection, first on GPT-4 and then on Mixture-of-Experts, RNN-like and State Space Models. The exact replication confirms the high-recall, low-precision pattern on GPT-4. The differentiated replication finds the pattern does not transfer uniformly: GPT-4 and Gemma-3 behave alike despite a roughly 100x parameter gap, while two MoE models diverge.
LLMTransformerMixture of Experts+3
Chen et al.GPT-4Gemma 3+1
research, evalsΒ· Added 30 Oct 2026, published 28 Oct 2026
This arXiv paper (October 2026) replicates Chen et al.'s study of LLM-based smart contract vulnerability detection. It confirms the high-recall, low-precision pattern on GPT-4, then tests Mixture-of-Experts and RNN-like models plus a preliminary State Space Model run. Gemma-3 matches GPT-4's profile despite far fewer parameters, while two Mixture-of-Experts models diverge, so architecture family alone does not predict detection behavior.
GPT-4GemmaLLM
Chen et al.
AI Security, AppSecΒ· Added 30 Oct 2026, published 28 Oct 2026
The paper studies detection and temporal localization of fraudulent inducements (prompts to follow, join chat groups, click links) in internet finance short videos. It compares OCR+LLM, OCR+MLLM and a zero-shot MLLM, and builds a dataset of 2,419 videos (66.49 hours, 3,194 events, four categories). OCR+MLLM reaches mIoU 26.52 versus 14.49 for keyframe OCR+LLM, with 67.0 percent macro recall, using a recall-weighted F2 localization reward.
MLLMLLMOCR
arXiv
research, multimodal, trainingΒ· Added 30 Oct 2026, published 28 Oct 2026
HerHealthGPT is a conversational AI system for women's reproductive health that combines retrieval-augmented generation, context-aware clarification and source validation in a web interface. Evidence retrieval raises concern-category accuracy from 54.1% to 59.3% over fine-tuning alone on 540 messages from 90 cases. The clarification router reaches 85.6% recall and 86.2% specificity. In a 30-adult study, 83.3% preferred the retrieval-enabled version. The authors say clinical validation is still needed.
RAGLLM
HerHealthGPTarXiv
research, llm-appsΒ· Added 27 Oct 2026, published 24 Oct 2026
DRILS is an LLM-based agentic framework for requirements change impact analysis. It uses multiple agents to generate candidate impact sets, with an LLM-as-a-judge promoting diversity in selection. It adds RE-RAG, a tuned retrieval technique, and a requirement dependency graph as a knowledge source. On three curated datasets it reaches 87.1% recall, 76.3% precision and 81% F1 at lower cost than baselines. The authors release the datasets.
LLMRAG
arXivDRILSRE-RAG
research, agents, llm-appsΒ· Added 27 Oct 2026, published 24 Oct 2026
RALE (Representation-Aware Logical Ensemble) is a method for LLM-based vulnerability detection in C code. The authors train LLMs on raw source code, Abstract Syntax Trees and Control Flow Graphs, and find each representation works best for certain vulnerability classes. Combining the models with explicit logical rules and majority voting raises F1 from at most 73% for a single model to 78%, with gains in accuracy, precision and recall.
LLMstatic analysis
arXivRALE
research, trainingΒ· Added 28 Oct 2026, published 24 Oct 2026
FORK is a reward for rubric-based reinforcement learning that jointly optimizes recall and a new precision objective. Precision requires each unit of a response to be necessary (rubric-driven necessity credits) and factually correct (a grader). Rubric-coverage rewards otherwise produce verbose, less accurate responses. FORK keeps similar coverage with shorter outputs, raising win rate against the base policy from 10.6% to 68.0% on RubricHub-Medical and from 22.1% to 74.1% on HealthBench.
LLM
HealthBenchRubricHub-MedicalFORK
training, researchΒ· Added 27 Oct 2026, published 24 Oct 2026
BARBICAN is a fail-closed prompt-injection gateway for MCP tool mediation. This paper shows its arbitration classifier ran on tokenizer-mangled text for a whole evaluation. Fixing it raised AUC from 0.392 to 0.855 while system recall fell from 80.67% to 80.34%. The routed band is 96.82% malicious, so deletion ablations mislead without substitution experiments and class-balance reporting.
Model Context ProtocolLLM
BARBICANprompt injection
AI SecurityΒ· Added 27 Oct 2026, published 23 Oct 2026
The paper analyses BARBICAN, a fail-closed prompt-injection gateway for MCP tool mediation, using deletion, substitution, repair and transplant experiments. Its arbitration classifier ran on tokenizer-mangled text for a whole evaluation. Fixing that raised AUC from 0.392 to 0.855 while system recall fell from 80.67% to 80.34%. The band it receives is 96.82% malicious. The authors argue ablation figures need the band's class balance and substitution baselines.
Model Context Protocol
BARBICAN
evals, research, mcpΒ· Added 27 Oct 2026, published 23 Oct 2026
This paper describes an evidence-aware retrieval evaluation protocol for a clinical RAG system over breast lesion management literature. Relevance is defined by evidence spans mapped to all containing chunks. A hybrid retriever (dense, BM25, cross-encoder reranking) with 384-token chunks and 20% overlap reached 0.94 Recall@3 on 18 queries. Expanding ground truth from 25 to 37 spans did not change rankings. The authors note the small set needs independent validation.
RAGBM25LLM
DeepSeekDeepEvalarXiv
llm-apps, evals, researchΒ· Added 24 Oct 2026, published 22 Oct 2026
Chunk Recomputation under Real Document Revisions measures how much text must be re-embedded after edits for three stateless chunkers (non-overlapping fixed-size, recursive paragraph-first, content-defined) on 120 pairs of adjacent Chinese and English Wikipedia revisions. Fixed-size recomputes 57-68% of the new-version workload, recursive 38-56%, content-defined 43-54%, while evidence recall at a 2048-token budget differs by under two points. The authors argue update cost should be reported alongside retrieval quality.
RAG
WikipediaarXiv
llm-apps, research, evalsΒ· Added 24 Oct 2026, published 22 Oct 2026
nokast-secureRAG is a roughly 0.5B-parameter small language model that works as a semantic firewall between retrieval and generation in local RAG systems. It classifies each query and context pair as safe, suspicious or malicious. It reaches 0.994 recall at a 0.026 false-positive rate in-distribution, beating Llama Guard 3 1B and Prompt-Guard-86M (0.900 recall, 0.690 FPR). Recall on PoisonedRAG falls to near zero, and single-request latency is 293-404 ms.
The paper separates "active" facts (which support generalization beyond recall) from "inert" ones. It builds new relations by relabeling answers of existing ones (Paris -> Europe becomes Paris -> crab), fine-tunes on a subset of subjects, and tests held-out subjects that share an answer (Munich -> crab). Across eleven language models and 70 relations, linear readability of the original relation in pretrained representations correlates with this generalization at Pearson r = 0.87 to 0.92 within each model. The result holds after controlling for fine-tuning set size, original-relation accuracy and subject frequency in pretraining text, and it survives re-randomized labels, seeds and thirteen other label categories. Mapping wrong labels back to the original answers shows the fine-tuned model's errors follow the linear probe's top answers.
LLM
arXiv
research, trainingΒ· Added 17 Oct 2026, published 15 Oct 2026
Presents Creed, an LLM-based code auditing approach that infers a developer's security-related beliefs for a function, scores how reasonable each is, and derives the conditions under which it is violated. It catches domain-specific semantic flaws that syntactic static analysis misses, such as a timing side-channel that undermines user-enumeration protection. The paper also introduces two measures for explanation-based vulnerability detection, recall and overhead. On a recent real-world benchmark, Creed beats existing explanation-based tools on the effectiveness-effort trade-off.
LLM
Creedtiming side-channeluser enumeration
AI Security, AppSecΒ· Added 15 Oct 2026, published 11 Oct 2026
Presents Creed, an LLM-based code auditing approach that, for a given function, infers the developer's security-related beliefs, scores how reasonable each belief is, and derives the conditions under which it is violated. This lets it find semantic flaws, such as timing side-channels in authentication, that syntactic static analysis tools miss, and it explains root cause, symptoms and exploitability. The paper also introduces two evaluation measures for explanation-based vulnerability detection: recall (how well a known vulnerability is correctly explained) and overhead (the effort of inspecting reports). On a recent real-world benchmark, Creed beats existing explanation-based tools on the effectiveness-effort trade-off.
LLMstatic analysis
CreedarXiv
research, evalsΒ· Added 15 Oct 2026, published 11 Oct 2026
This paper introduces Personal Fact Memory (PFM), a reference layer for diagnosing errors in language-agent memory systems before generation happens, separating stored-state validity, identity resolution, abstention and serving latency. On a controlled revision benchmark, serving only the active value per correctly keyed slot eliminates stale exposure (versus 70.3 percent stale exposure without update resolution), while participant-aware BM25 matches a reference ranker within a 0.02 margin once stores are shared. The harder problem is revision-to-slot assignment: four LLM key assigners get higher key recall than a rule-based extractor but lower clean-retrieval rates, open-domain merge recall on LongMemEval never exceeds 0.062, and entity-posterior filtering reduces but does not eliminate same-name misattribution on LoCoMo; prompt prefill, not retrieval, dominates turn-level latency.
BM25
LongMemEvalLoCoMo
agents, evalsΒ· Added 8 Oct 2026, published 7 Oct 2026
This paper runs a mechanism-isolating ablation of four document-structure treatments used in retrieval-augmented generation β structure-aligned chunking, LLM-generated chunk contexts, heading-path metadata, and hierarchical two-stage retrieval β under one protocol with matched chunk sizes and a semantically null placebo (shuffled heading paths) to control for the confound that any prepended text perturbs embeddings. Using a coverage-aware nDCG metric and document-clustered bootstrap with Holm correction across 200 Wikipedia Featured Articles (951 queries) and 1,585 QASPER papers (4,303 questions), it finds that structure-aligned chunking with real heading paths beats both contextualized fixed windows and the placebo, naive hierarchical retrieval hurts performance (traced to first-stage section recall failures), and gold structure beats LLM-induced structure on Wikipedia but not QASPER. Effect sizes are small but statistically significant and consistent across corpora, giving RAG practitioners evidence-based guidance on which structure-aware retrieval tricks actually help versus which are placebo effects.
QASPERWikipedia Featured Articles
research, llm-appsΒ· Added 8 Oct 2026, published 7 Oct 2026
UNREAL is a model-native evidence-selection framework that unifies corpus retrieval and long-context inference using a single mechanism built on top of a frozen LLM, adding fewer than 500K trainable parameters. It derives retrieval queries and chunk encodings directly from the LLM's internal representations, outperforming retriever-reranker systems on a 21M-chunk Wikipedia index (e.g. HotpotQA recall 49.1%β73.2%) and improving long-context accuracy (NoLiMa 1.0%β24.83% at 128K tokens) while cutting FLOPs and time-to-first-token versus full-context inference.
TransformerBM25
HotpotQA2WikiMultiHopQAMuSiQue+2
research, llm-appsΒ· Added 7 Oct 2026, published 6 Oct 2026
Agentic-ZTA is a multi-agent architecture that operationalizes the NIST SP 800-207 Zero Trust Architecture control loop using coordinated AI agents. A Policy Enforcement Point intercepts access requests, enriches them with context, and routes them to a policy-engine agent that invokes domain-specialized core and supporting agents; policies are retrieved via RAG and embedded into agent prompts at inference time, with trust scores aggregated by a trust algorithm to produce the final access decision. The authors implement and evaluate the framework on a testbed, reporting 95.0% accuracy, 93.9% precision, and 96.3% recall on representative access-control scenarios.
LLM
NIST SP 800-207
Identity & Access, AI SecurityΒ· Added 6 Oct 2026, published 5 Oct 2026
This IACR ePrint paper presents a Linux runtime cryptographic discovery system built on eBPF user-space probes that detects both classical and post-quantum (ML-KEM) cryptographic operations as they execute, complementing static binary/SBOM analysis with dynamic observation. It uses "randomness anchoring" β correlating observed key-generation/encapsulation randomness with subsequent crypto API calls in the same process β to raise detection confidence without adding false positives, reporting perfect precision/recall on classical and ML-KEM workloads, median detection latency of 28 ΞΌs, and 14β18% overhead on key-generation-bound workloads. A nine-application case study (drawn from/paired with QED datasets) compares static vs. runtime views and shows how eBPF probe coverage limits what can be observed, relevant to building a runtime cryptographic bill of materials for PQC migration compliance.
eBPFLinuxML-KEM+1
QED datasets
AppSec, GRCΒ· Added 8 Oct 2026, published 5 Oct 2026
Physis-Lang is a self-evolving framework that uses natural language (describing entities, causes, interactions, governing principles, and temporal evolution) as the shared physical-knowledge representation across data curation, training, and generation for video world models. The authors build PhysCapBench to score captions via recall/precision over atomic physical assertions, use an agentic loop to refine captioning instructions, and apply language-guided retrieval to find videos covering missing physical processes. Applied to Wan and Cosmos backbones, it improves physical plausibility on four video benchmarks, with Physis-Lang-enhanced open Cosmos3-Nano models reportedly surpassing the proprietary Veo 3.1.
WanCosmosCosmos3-Nano+2
research, multimodalΒ· Added 1 Oct 2026, published 30 Sep 2026
This paper shows that adding an adversarial loss as a post-training correction to pretrained pixel diffusion/flow-matching models (applied at non-high-noise timesteps, architecture and sampler unchanged) restores missing high-frequency, fine-scale natural-image statistics that these models systematically underproduce. Through frequency-band/power-law analysis and nearest-neighbor, recall, and no-GAN matched-SFT controls, the authors rule out memorization or mode-dropping as explanations, and show the effect jointly improves distribution fidelity, coverage, prompt alignment and perceptual quality β but does not transfer comparably to latent diffusion configurations, pointing to direct pixel-output access as the key enabling factor.
diffusion modelflow matchingGAN+2
research, multimodalΒ· Added 30 Sep 2026, published 29 Sep 2026
SmartMemory is an academic framework that detects "OFC inconsistency" (OFCI) β a newly identified root-cause vulnerability class where business-logic flaws break the equivalence between a smart contract's on-chain state and its off-chain counterpart, affecting cross-chain bridges, real-world-asset tokenization, and fiat-backed stablecoins. It normalizes heterogeneous on-chain/off-chain implementations into a canonical business-semantic representation, uses a memory-based agent to distill and reuse vulnerability patterns across cases, and verifies candidates with taint analysis. Evaluated on a new dataset of 48 DApps with 81 known OFCIs (80.68% precision, 87.65% recall), it also found 36 previously unknown, since-fixed OFCIs in an analysis of 325 real-world OFC applications.
Blockchain, AppSecΒ· Added 2 Oct 2026, published 28 Sep 2026
This arXiv paper proposes mining cyber-physical invariants (conservation laws, actuator couplings) from clean operational data to serve as a verifiable admission gate for federated learning updates in industrial intrusion-detection systems, closing a blind spot where poisoned updates derived from fabricated telemetry are indistinguishable from honest ones in update space alone. Evaluated across SWaT, WADI, and BATADAL water-infrastructure testbeds against seven aggregation rules, the gate rejects all fabricated shards (including ones that defeat FoolsGold) while admitting all honest ones, and recovers most of the detection recall lost to replay and fabrication poisoning. The authors also show invariant compliance can be proven via zk-SNARKs so clients avoid revealing raw operational telemetry.
zk-SNARKs
SWaTWADIBATADAL+1
AI SecurityΒ· Added 2 Oct 2026, published 28 Sep 2026
Re:CAP (REtrieval Coverage Audit by iterative Probing) is a reference-free method for auditing retrieval coverage in production RAG systems where exhaustive relevance labels don't exist. It iteratively probes a deployed pipeline's answer and retrieved context for plausibly missing topics, retrieves candidates, and uses an LLM-as-judge to keep only documents that add previously-unretrieved information. On four public benchmarks it recovers 9-48% of gold documents that flat BM25/hybrid top-500 retrieval misses (up to 48% on TREC-COVID), beats a hybrid baseline by +12.9pp recall on MuSiQue at half the document budget, and is validated by human annotators (Fleiss ΞΊ = 0.79) on both benchmark and live production traffic.
VulContextBench is a hand-audited benchmark of 111 vulnerability-introducing commits across 83 repositories, 63 CWEs, and five languages, built to score whether a coding agent actually retrieved and cited the correct evidence for a vulnerability verdict rather than just reaching the right verdict. Evaluating seven frontier models (including Qwen3-Coder-Next and GPT-5.5) at file/block/line granularity, the authors find every model explores most of the gold evidence but reports only a fraction of it: Qwen3-Coder-Next views 86.3% of annotated lines but cites just 12.9%, while GPT-5.5 views 73% and reports 36%. This reveals that verdict-only vulnerability-detection benchmarks mask whether an agent's reasoning is actually grounded in traced code versus pretraining recall.
Qwen
Qwen3-Coder-NextGPT-5.5VulContextBench
AI Security, AppSecΒ· Added 2 Oct 2026, published 26 Sep 2026
VulContextBench is a benchmark of 111 hand-audited vulnerability-introducing commits across 83 repositories, 63 CWEs and five languages, built to measure whether a coding agent's verdict is actually grounded in the code it retrieved rather than recalled from pretraining. Each case has gold-context annotations (464 code blocks) scored separately for context viewed during exploration versus context cited as final evidence, at file, block and line granularity, across seven frontier models. The headline finding is a large viewed-vs-reported gap β e.g. Qwen3-Coder-Next views 86.3% of gold lines but cites only 12.9%, GPT-5.5 views 73% and reports 36% β showing agents explore relevant evidence but fail to surface it in their final report, a failure mode invisible to verdict-only benchmarks.
Qwen3-Coder-NextGPT-5.5
VulContextBench
evals, coding-agentsΒ· Added 2 Oct 2026, published 26 Sep 2026
SCAR is an adaptive retrieval policy for RAG that combats boundary fragmentation from fixed-length chunking by selectively expanding neighboring chunks, using a relative expansion threshold tied to each chunk's own query-relevance rather than a fixed window size. Tested on four corpora (RFC, GDPR, a 10-K filing, a merger agreement; 320 queries) it hits 92.8% recall on boundary-fragmented queries using 22.9% fewer chunks than static windowing, transfers across three embedding models (text-embedding-3-large, BGE-large-en-v1.5, zembed-1) without recalibration, and cuts context tokens by 27.1% while preserving RAGAS-measured generation faithfulness.
CallScreenBench is a benchmark for evaluating small, quantized (4-bit, 0.6-4B parameter) language models acting as autonomous phone secretaries that screen inbound calls without a cooperative caller-assigned task. It reports five automated call-and-note measure groups tied to owner endorsement (service, recall, plausibility, triage discrimination) plus three guardedness diagnostics for toolless, credential-free proxies, deliberately avoiding a single composite score to expose trade-offs like scam-detection TPR versus legitimate-call false positives. Testing across three paired model-family checkpoints reveals that larger checkpoints score higher on some measures while triage discrimination follows a different ordering, and scripted degenerate policies (e.g., hangup-and-echo) expose measurement floors.
This paper proposes no-box vulnerability analysis, a new paradigm for auditing systems using only functionality metadata when neither system access nor runtime interaction is available. The authors build MCPSEC, a prototype that audits Model Context Protocol servers for indirect prompt injection vulnerabilities purely from tool metadata exposed at server registration, evaluating it against 20 widely-deployed MCP servers with 177 tools where human evaluators confirmed 95 vulnerable tools; MCPSEC achieved 98.9% recall versus 84.2% for an LLM baseline, also producing hypothesized exploitation techniques per finding.
arxiv.orgpaperπ§ AI
