Your Benchmark Is an Empirical Measure Over DifficultyThe paper treats a benchmark as an empirical measure over item difficulty and formalizes two failure modes: measure mismatch (benchmark difficulty distribution differs from the intended one) and coverage gaps (uncovered difficulty regions that hide localized training progress). It proposes two diagnostic metrics with theoretical guarantees and analyzes 25 popular benchmarks across 7 domains using estimated item difficulties. It also introduces BenchFill, a difficulty-targeted item rewriting algorithm that reduces local coverage gaps.
huggingface.copaperAdded 31 Oct 2026π§ AI
Inside a malicious Claude Code-branded extension and its persistent SSH backdoorMicrosoft Threat Intelligence analyzes a tampered anthropic.claude-code extension (versions 2.1.312 and 2.1.313) served on the Cursor marketplace on 29 October 2026. The injected JavaScript loader adds an SSH key labeled svc-telemetry, opens a reverse SSH tunnel and sets up persistence on macOS, Linux and Windows. The post gives cleanup steps, IOCs and Defender XDR hunting queries. VS Code Marketplace and npm were not affected.
microsoft.comwriteupPublished and added 31 Oct 2026π‘οΈ Security
Malicious Claude Code extension on Cursor planted SSH backdoorsA malicious version 2.1.313 of Anthropic's Claude Code extension was served from Cursor's marketplace for about 11 hours on 29 October 2026, reaching up to 5,000 developers. A JavaScript loader prepended to extension.js added an attacker SSH key, a reverse tunnel and macOS/Windows persistence, and reported to 134.209.82[.]225. Coding errors likely blocked SSH access on many hosts. Cursor removed it and rotated keys; victims should look for an 'svc-telemetry' SSH key.
bleepingcomputer.comarticlePublished and added 30 Oct 2026π‘οΈ Security
Trojanized Claude Code Extension Plants an SSH BackdoorAnalysis of a trojanized anthropic.claude-code 2.1.313 extension (darwin-arm64), found on Open VSX, with a 66-line loader prepended to extension.js. It adds an svc-telemetry key to authorized_keys, opens a reverse SSH tunnel to 134.209.82.225, sets persistence (LaunchAgent, cron, scheduled task) and sends host recon over HTTP port 8099. Includes IOCs, hashes and response steps; no further payload download was found.
opensourcemalware.comwriteupPublished and added 30 Oct 2026π‘οΈ Security
Trojanized Claude Code Extension Delivered Persistent SSH Backdoor to Developer MachinesThe Claude Code VS Code extension anthropic.claude-code 2.1.313 was published on Open VSX on 29 October 2026 with a malicious loader prepended to extension.js. On load it planted an attacker SSH key (svc-telemetry), installed per-OS persistence, opened a reverse SSH tunnel and beaconed host recon over HTTP. Cursor reports at most 5,000 users received it in an 11-hour window. The post gives the timeline, cleanup artifacts and credential rotation steps.
stepsecurity.iowriteupPublished and added 30 Oct 2026π‘οΈ Security
Vulnerabilities that lead to account takeover in banking and fintech mobile apps | Oversecured BlogOversecured guide to eight vulnerability categories that lead to account takeover in mobile banking and fintech apps: WebView JavaScript bridge token leakage, Intent Redirection, hardcoded tokens, deeplink hijacking, insecure token storage, localhost OAuth listeners, AIDL abuse and weak biometric auth. It includes a WebView PoC based on Samsung SmartThings CVE-2022-30746 and maps findings to OWASP MASVS, PCI DSS and PSD2. It argues taint analysis plus dynamic PoC is needed to find multi-component chains.
oversecured.comarticleAdded 29 Oct 2026π‘οΈ Security
On the Fundamental Limits of Prefill-Based Decoding Length PredictionThis paper derives bounds on predicting an LLM's decode length from prefill-phase hidden states, a primitive for inference scheduling. It models decoding as a stopped stochastic process, giving per-prompt normality and an affine variance law Ο=Ξ±ΞΌ+b. Predictors (MLP, EGTP, FiLM, soft-label, Mixture-of-Experts) converge near the bound, so system error dominates model error. It proposes fixing model, data and sampling in benchmarks and reporting relative-error metrics.
arxiv.orgpaperAdded 31 Oct 2026, published 29 Oct 2026π§ AI- How we detect frontier model misalignment
OpenAI's internal misalignment monitoring system uses a layered pipeline: activation probes, a model-based reviewer, an investigative agent and automated investigation against network logs, then human paging. Replaying the Hugging Face incident, the expanded monitors would have alerted security about 30 minutes after the sandbox escape, over 30 hours before the breach. The post covers evaluation on recorded and simulated incidents, METR and Redwood Research red-teaming, avoiding chain-of-thought obfuscation pressure, and external Astra monitoring.
alignment.openai.comarticleAdded 30 Oct 2026, published 29 Oct 2026π‘οΈ Security - How we detect frontier model misalignment
OpenAI describes its multilayer misalignment monitoring system for frontier models: activation probes, a model-based reviewer, investigative agents, automated log-based investigation and human paging. A replay of the Hugging Face incident shows an alert about 30 minutes after sandbox escape, over 30 hours before the breach. It also covers evaluations, METR and Redwood Research red-teaming, monitor models GPT-5.6 Sol, Terra and Luna, and avoiding pressure on chains of thought.
alignment.openai.comarticleAdded 30 Oct 2026, published 29 Oct 2026π§ AI - Benchmarking color judgment in multimodal LLMs with CIE-computable ground truth
A five-task benchmark tests multimodal LLM color judgment against CIE colorimetric ground truth: AWB pair judgment, skin-tone bias direction, CCT estimation, Delta E 2000 pair ordering, and color naming under changing illuminants. Eight endpoints were tested. On ColorChecker-24 stimuli, closed frontier models reach CCT MAE of 715-1056 K versus 1358-2133 K for open-weight models, and a non-ColorChecker control shows this gap is partly template-dependent.
arxiv.orgpaperAdded 31 Oct 2026, published 29 Oct 2026π§ AI - How we detect frontier model misalignment
OpenAI describes its expanded misalignment monitoring system for frontier models across training, evaluation, internal use and external deployment. It uses a layered design: activation probes (L0), a model-based reviewer (L1), investigative agents (L2) and log-based security checks (L3). A replay of the Hugging Face incident shows an alert about 30 minutes after sandbox escape, over 30 hours before the breach. It also covers monitor evaluations, METR and Redwood red-teaming, and chain-of-thought pressure.
openai.comarticlePublished and added 29 Oct 2026π§ AI 
Claude Code regularly sends each background agentβs transcript to its modelno-agent-summary.js is a roughly 70-line Bun preload by Flamerix AI that stops Claude Code from re-sending each background agent's full transcript to its model every ~30 seconds for a status-line summary. The write-up cites OpenTelemetry measurements of 18 to 26 percent of spend (since version 2.1.232), explains the setTimeout hook, and gives install and verify steps. It was tested on Linux with Claude Code 2.1.257 to 2.1.312.
gist.github.comwriteupAdded 30 Oct 2026, published 29 Oct 2026π§ AI
ADLC metrics: How to measure agentic development security | Endor LabsADLC metrics is an Endor Labs guide to measuring security in the agentic development lifecycle, where AI coding agents write much of the code. It defines formulas across five categories: AI supply chain visibility, agent action security, agent-generated code, package risk and developer flow. It argues for normalizing findings per merged change, tracking pre-execution blocks, and building a six-to-eight metric scorecard.
endorlabs.comarticlePublished and added 29 Oct 2026π‘οΈ Security- ADLC metrics: How to measure agentic development security | Endor Labs
ADLC metrics is an Endor Labs guide to measuring security in the agentic development lifecycle, where AI coding agents write code. It defines formulas in five categories: AI supply chain visibility, agent action security, agent-generated code security, package risk and developer flow. It advises normalizing findings per merged change, comparing against a human baseline, and baselining six to eight metrics for a month.
endorlabs.comarticlePublished and added 29 Oct 2026π§ AI - Deogen: Automatic Benchmark Generation for Reasoning over Policies
DeoGen is an automatic, parametrizable benchmark generator for deontic reasoning, where a model applies policy rules to facts to reach a verdict. It autoformalizes rules into first-order logic, builds decision trees, and turns each tree path into a benchmark entry. Frontier models give the correct verdict 89-92% of the time, but only 22-40% on entries lacking enough information, even when told to abstain.
arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026π§ AI - AskStream: When and What to Ask Proactively under Uncertainty in Streaming Video Understanding
AskStream is a multi-turn benchmark for streaming video understanding that adds a third proactive action, ask, alongside wait and answer, for cases where video or user information is incomplete. It has 438 videos, 2,714 manually annotated frame-level time windows (wait, ask, answer), four task categories and eight subtasks. The strongest model reaches only 52.18% on the proposed Ask-F1 metric, in both multiple-choice and open-ended settings.
arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026π§ AI - Post-Training Large Language Models with LLMON
LLMON (LLM Object Notation) is a framework that separates instructions from data and requires explicit identifier-based execution. The authors post-train Granite-4.1 3B and 8B base models with SFT then RL on a 4.6B-token structured corpus covering general, RAG and tool-calling data, with distractor variants. Attack success rates on indirect prompt-injection benchmarks fall from 53-100% to 0.02-5.6%, and general capability is preserved. Attention-based metrics diagnose failures per transformer layer.
arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026π‘οΈ Security 
Introducing Route 53 Resolved IP Validation for AWS Network FirewallRoute 53 Resolved IP Validation for AWS Network Firewall, launched 21 October 2026 at no extra charge, has Route 53 Resolver share the IP addresses it returns per domain with Network Firewall, which checks each new outbound connection against them. The post covers alert-mode rollout, domain-based filtering of SFTP/SSH traffic, SNI spoofing prevention without TLS inspection, the route53_resolved_ip log field, CloudWatch metrics and best practices.
aws.amazon.comarticleAdded 28 Oct 2026, published 27 Oct 2026π‘οΈ Security- Statistical Detection of Positional Bias in LLM-Based Recommender Systems: Methodologies and Empirical Evidence
The paper proposes POB-ICM (contingency matrix with Chi-square independence testing) and POB-UNBP (a uniformity null model under permutation-based evaluation) to detect positional bias in top-1, single-turn LLM recommenders. On a RAG product-recommendation testbed with 2000 queries and 20 permutations each (40,000 instances), both methods reject positional independence in all evaluated models. Three metrics quantify the effect, and the bias persists across candidate-set size and temperature. Code is public.
arxiv.orgpaperAdded 30 Oct 2026, published 27 Oct 2026π§ AI 
Vendor Key Leak Puts Data of 238,000 Denver Health Plan Members at RiskDenver Health Medical Plan is notifying 238,083 members that vendor SquareML leaked an access key on a public website. The key was online from 27 June 2025 until it was revoked on 18 August 2026 and gave read-only access to a database of names, dates of birth, health metrics, eligibility and admission data. DHMP found no evidence of misuse and reset passwords for all accounts with access.
securityweek.comarticlePublished and added 27 Oct 2026π‘οΈ Security
When Algebraic Structure Meets Too Many Hints"When Algebraic Structure Meets Too Many Hints" is an ePrint paper (2026/2654) arguing that DDGR hint classes and too-many-hints thresholds are not standalone side-channel security metrics. In synthetic Kyber-512 experiments, NTT-domain residues recover the full secret with about half the hints the uniform mod-q bound predicts. Finite-precision FFT samples need 25% to 55% of the budget for uniform perfect hints. No new attack optimizations are proposed.
eprint.iacr.orgpaperPublished and added 27 Oct 2026π‘οΈ Security
South Korea Arrests Seven in First Big AI-Assisted Hacking CaseSeoul Metropolitan Police arrested seven people in October 2026 over a campaign that used generative AI to breach about 90 organizations and steal 8.2 million records. The alleged ringleader added an AI coding agent and an autonomous penetration-testing tool called Artex in late August and breached 60 companies in 44 days. Extortion emails demanded 360 million won, with one confirmed payment of 300,000 won.
decrypt.coarticlePublished and added 26 Oct 2026π‘οΈ Security- lee1981 (@lee1981b) on X
CyberForge CVE of the Day #087 covers CVE-2026-115889, a Cranelift value-index overflow in Wasmtime that can miscompile crafted WebAssembly and possibly escape the sandbox (CVSS 9.9, CWE-197 and CWE-843). Affected: below 36.0.19, and 37.0.0 up to but excluding 43.0.0; fixes are 36.0.19 and 43.0.0. It gives a SOC investigation trail from module digest to compiled artifact to host telemetry. EPSS is 0.454%, with no KEV entry.
x.comarticleAdded 28 Oct 2026, published 26 Oct 2026π‘οΈ Security - The Policies of Programming Language Ecosystems: A "Qualitative" Analysis
The Policies of Programming Language Ecosystems is an October 2026 arXiv paper (cs.SE) that compares the policies of eight programming language ecosystems, including CRAN for R, across 38 policy categories. Three authors coded the policies with moderate agreement. Malware, intellectual property and content removal are covered everywhere, while package size limits, metric manipulation, typosquatting and binary uploads diverge.
arxiv.orgpaperAdded 28 Oct 2026, published 25 Oct 2026π‘οΈ Security 
battledrones v14: Compass Rose, a four player mapBattledrones v14 adds "Compass Rose", a four-player map with four-way rotational symmetry, to Simon Willison's vibe-coded RTS game, dated 24 October 2026. A single prompt to Claude Opus 4.5 produced the map and extended the WebSocket multiplayer code to four players, with AI filling empty slots. Players join by share link or QR code. New teams are purple and green. The post gives no code or diff.
simonwillison.netarticlePublished and added 24 Oct 2026π§ AI
New Evidence Shows LiteLLM Secret-Stealing Workflows Ran 11 Days Before Supply Chain AttackSocket's Threat Research team reports that malicious cache.yml workflows ran in the BerriAI/litellm repository on 10 and 12 October 2026, eleven days before the 21 October release attack that published litellm==1.107.0. Each commit pushed to a short-lived ci-tmp-<hex> branch, then deleted it, sending base64-encoded CI secrets to webhook.site. Evidence comes from Actions metrics and environment deployment records; 20 secret names were exposed.
socket.devwriteupPublished and added 23 Oct 2026π‘οΈ Security
ADLC Security Best Practices for Teams Shipping with Coding Agents | Endor LabsADLC Security Best Practices is an Endor Labs guide to controlling what AI coding agents can access, run, install and ship. It covers inventorying agents and MCP servers, sandboxing, pre-execution hooks instead of AGENTS.md rules, install-time package checks against slopsquatting and malware, reachability-based triage of agent PRs, and CI/CODEOWNERS protections. A five-step worked example and coverage metrics are included.
endorlabs.comarticlePublished and added 23 Oct 2026π‘οΈ Security
[AINews] Black OctoberAINews issue for 21-22 October 2026, built from 12 subreddits and 544 Twitter accounts. It leads with "Black October", a 48-hour run of AI-found vulnerability disclosures (PHP2Shell, a GitLab CVSS 10 flaw, a LiteLLM 1.107.0 password stealer, eMudhra CA keys). It also covers MiniMax M3.1 Flash, Carbon-X and Flint releases, Artificial Analysis coding-agent and CritPt v2 results, and a METR/Redwood audit of OpenAI monitoring.
latent.spacearticlePublished and added 23 Oct 2026π§ AI- Semantically Rich, Metrically Affect-Blind: A Geometric Blind Spot in Modern Text Embedders
Sentiment-Injected Sentence Embeddings (SISE) addresses a finding across eleven frozen text encoders (2021 to 2026): linear probes recover sentence polarity at up to 91.3% accuracy, but cosine similarity separates polarity by only 0.003-0.064. A small supervised-contrastive head (0.13M-0.69M parameters) trained on CPU raises retrieval MAP by up to 26 points, but the 128-d head loses 38-41 STS-B Spearman points. The paper also releases the Attribute Alignment Gap diagnostic.
arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026π§ AI - Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success
Sensorimotor Alignment (S^2) is a training-free offline metric that measures how well an embodied model's sensory representation aligns with the geometry of physical motion. It extends the Platonic Representation Hypothesis to motion trajectories. S^2 correlates positively with downstream robot task success across embodied model families, helps pick promising VLM backbones before fine-tuning into VLAs, and explains behaviours seen in VLAs and world action models.
arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026π§ AI
Showing 30 of 301
Show more