The paper treats a benchmark as an empirical measure over item difficulty and formalizes two failure modes: measure mismatch (benchmark difficulty distribution differs from the intended one) and coverage gaps (uncovered difficulty regions that hide localized training progress). It proposes two diagnostic metrics with theoretical guarantees and analyzes 25 popular benchmarks across 7 domains using estimated item difficulties. It also introduces BenchFill, a difficulty-targeted item rewriting algorithm that reduces local coverage gaps.
Microsoft Threat Intelligence analyzes a tampered anthropic.claude-code extension (versions 2.1.312 and 2.1.313) served on the Cursor marketplace on 29 October 2026. The injected JavaScript loader adds an SSH key labeled svc-telemetry, opens a reverse SSH tunnel and sets up persistence on macOS, Linux and Windows. The post gives cleanup steps, IOCs and Defender XDR hunting queries. VS Code Marketplace and npm were not affected.
A malicious version 2.1.313 of Anthropic's Claude Code extension was served from Cursor's marketplace for about 11 hours on 29 October 2026, reaching up to 5,000 developers. A JavaScript loader prepended to extension.js added an attacker SSH key, a reverse tunnel and macOS/Windows persistence, and reported to 134.209.82[.]225. Coding errors likely blocked SSH access on many hosts. Cursor removed it and rotated keys; victims should look for an 'svc-telemetry' SSH key.
Claude CodeCursorVS Code+3
PhorionOpenSourceMalwareLaunchAgent persistence+1
Supply Chain, AI Security, Threat IntelΒ· Published and added 30 Oct 2026
Analysis of a trojanized anthropic.claude-code 2.1.313 extension (darwin-arm64), found on Open VSX, with a 66-line loader prepended to extension.js. It adds an svc-telemetry key to authorized_keys, opens a reverse SSH tunnel to 134.209.82.225, sets persistence (LaunchAgent, cron, scheduled task) and sends host recon over HTTP port 8099. Includes IOCs, hashes and response steps; no further payload download was found.
The Claude Code VS Code extension anthropic.claude-code 2.1.313 was published on Open VSX on 29 October 2026 with a malicious loader prepended to extension.js. On load it planted an attacker SSH key (svc-telemetry), installed per-OS persistence, opened a reverse SSH tunnel and beaconed host recon over HTTP. Cursor reports at most 5,000 users received it in an 11-hour window. The post gives the timeline, cleanup artifacts and credential rotation steps.
Oversecured guide to eight vulnerability categories that lead to account takeover in mobile banking and fintech apps: WebView JavaScript bridge token leakage, Intent Redirection, hardcoded tokens, deeplink hijacking, insecure token storage, localhost OAuth listeners, AIDL abuse and weak biometric auth. It includes a WebView PoC based on Samsung SmartThings CVE-2022-30746 and maps findings to OWASP MASVS, PCI DSS and PSD2. It argues taint analysis plus dynamic PoC is needed to find multi-component chains.
This paper derives bounds on predicting an LLM's decode length from prefill-phase hidden states, a primitive for inference scheduling. It models decoding as a stopped stochastic process, giving per-prompt normality and an affine variance law Ο=Ξ±ΞΌ+b. Predictors (MLP, EGTP, FiLM, soft-label, Mixture-of-Experts) converge near the bound, so system error dominates model error. It proposes fixing model, data and sampling in benchmarks and reporting relative-error metrics.
LLMMixture of ExpertsMLP+2
arXiv
research, inference, evalsΒ· Added 31 Oct 2026, published 29 Oct 2026
OpenAI's internal misalignment monitoring system uses a layered pipeline: activation probes, a model-based reviewer, an investigative agent and automated investigation against network logs, then human paging. Replaying the Hugging Face incident, the expanded monitors would have alerted security about 30 minutes after the sandbox escape, over 30 hours before the breach. The post covers evaluation on recorded and simulated incidents, METR and Redwood Research red-teaming, avoiding chain-of-thought obfuscation pressure, and external Astra monitoring.
OpenAICodexHugging Face+3
sandbox escapeMETRRedwood Research+2
AI Security, Blue Team & DetectionΒ· Added 30 Oct 2026, published 29 Oct 2026
OpenAI describes its multilayer misalignment monitoring system for frontier models: activation probes, a model-based reviewer, investigative agents, automated log-based investigation and human paging. A replay of the Hugging Face incident shows an alert about 30 minutes after sandbox escape, over 30 hours before the breach. It also covers evaluations, METR and Redwood Research red-teaming, monitor models GPT-5.6 Sol, Terra and Luna, and avoiding pressure on chains of thought.
CodexOpenAI APIChatGPT+2
OpenAIMETRRedwood Research+9
ai-safety, agents, evalsΒ· Added 30 Oct 2026, published 29 Oct 2026
A five-task benchmark tests multimodal LLM color judgment against CIE colorimetric ground truth: AWB pair judgment, skin-tone bias direction, CCT estimation, Delta E 2000 pair ordering, and color naming under changing illuminants. Eight endpoints were tested. On ColorChecker-24 stimuli, closed frontier models reach CCT MAE of 715-1056 K versus 1358-2133 K for open-weight models, and a non-ColorChecker control shows this gap is partly template-dependent.
vision-language modelQwen3-VLGLM+2
GLM-4.5VGPT-5.4Claude-Opus-4.7+4
evals, multimodal, researchΒ· Added 31 Oct 2026, published 29 Oct 2026
OpenAI describes its expanded misalignment monitoring system for frontier models across training, evaluation, internal use and external deployment. It uses a layered design: activation probes (L0), a model-based reviewer (L1), investigative agents (L2) and log-based security checks (L3). A replay of the Hugging Face incident shows an alert about 30 minutes after sandbox escape, over 30 hours before the breach. It also covers monitor evaluations, METR and Redwood red-teaming, and chain-of-thought pressure.
CodexOpenAI APIChatGPT+1
OpenAIGPT-6 AstraGPT-5.5+9
ai-safety, agents, evalsΒ· Published and added 29 Oct 2026
no-agent-summary.js is a roughly 70-line Bun preload by Flamerix AI that stops Claude Code from re-sending each background agent's full transcript to its model every ~30 seconds for a status-line summary. The write-up cites OpenTelemetry measurements of 18 to 26 percent of spend (since version 2.1.232), explains the setTimeout hook, and gives install and verify steps. It was tested on Linux with Claude Code 2.1.257 to 2.1.312.
Claude CodeBunOpenTelemetry+2
AnthropicFlamerix AIDRIM+1
coding-agents, ai-infraΒ· Added 30 Oct 2026, published 29 Oct 2026
ADLC metrics is an Endor Labs guide to measuring security in the agentic development lifecycle, where AI coding agents write much of the code. It defines formulas across five categories: AI supply chain visibility, agent action security, agent-generated code, package risk and developer flow. It argues for normalizing findings per merged change, tracking pre-execution blocks, and building a six-to-eight metric scorecard.
Claude CodeCodexCursor+3
ADLCOWASP Top 10 for LLM ApplicationsAgent Security League+4
AI Security, AppSec, DevSecOpsΒ· Published and added 29 Oct 2026
ADLC metrics is an Endor Labs guide to measuring security in the agentic development lifecycle, where AI coding agents write code. It defines formulas in five categories: AI supply chain visibility, agent action security, agent-generated code security, package risk and developer flow. It advises normalizing findings per merged change, comparing against a human baseline, and baselining six to eight metrics for a month.
Claude CodeCodexCursor+4
Endor LabsAURIOWASP+4
coding-agents, ai-policyΒ· Published and added 29 Oct 2026
DeoGen is an automatic, parametrizable benchmark generator for deontic reasoning, where a model applies policy rules to facts to reach a verdict. It autoformalizes rules into first-order logic, builds decision trees, and turns each tree path into a benchmark entry. Frontier models give the correct verdict 89-92% of the time, but only 22-40% on entries lacking enough information, even when told to abstain.
LLMfirst-order logic
DeoGenarXiv
evals, researchΒ· Added 30 Oct 2026, published 28 Oct 2026
AskStream is a multi-turn benchmark for streaming video understanding that adds a third proactive action, ask, alongside wait and answer, for cases where video or user information is incomplete. It has 438 videos, 2,714 manually annotated frame-level time windows (wait, ask, answer), four task categories and eight subtasks. The strongest model reaches only 52.18% on the proposed Ask-F1 metric, in both multiple-choice and open-ended settings.
multimodal large language model
AskStreamAsk-F1arXiv
evals, multimodal, researchΒ· Added 30 Oct 2026, published 28 Oct 2026
LLMON (LLM Object Notation) is a framework that separates instructions from data and requires explicit identifier-based execution. The authors post-train Granite-4.1 3B and 8B base models with SFT then RL on a 4.6B-token structured corpus covering general, RAG and tool-calling data, with distractor variants. Attack success rates on indirect prompt-injection benchmarks fall from 53-100% to 0.02-5.6%, and general capability is preserved. Attention-based metrics diagnose failures per transformer layer.
LLMONGranite 4.1LLM
prompt injectionindirect prompt injection
AI SecurityΒ· Added 30 Oct 2026, published 28 Oct 2026
Route 53 Resolved IP Validation for AWS Network Firewall, launched 21 October 2026 at no extra charge, has Route 53 Resolver share the IP addresses it returns per domain with Network Firewall, which checks each new outbound connection against them. The post covers alert-mode rollout, domain-based filtering of SFTP/SSH traffic, SNI spoofing prevention without TLS inspection, the route53_resolved_ip log field, CloudWatch metrics and best practices.
AWS Network FirewallAmazon Route 53 ResolverRoute 53 Resolver DNS Firewall+3
SNI spoofingRoute 53 Resolved IP ValidationDNS tunneling
Cloud & Infra, Blue Team & DetectionΒ· Added 28 Oct 2026, published 27 Oct 2026
The paper proposes POB-ICM (contingency matrix with Chi-square independence testing) and POB-UNBP (a uniformity null model under permutation-based evaluation) to detect positional bias in top-1, single-turn LLM recommenders. On a RAG product-recommendation testbed with 2000 queries and 20 permutations each (40,000 instances), both methods reject positional independence in all evaluated models. Three metrics quantify the effect, and the bias persists across candidate-set size and temperature. Code is public.
LLMRAG
arXivPOB-ICMPOB-UNBP
research, evals, llm-appsΒ· Added 30 Oct 2026, published 27 Oct 2026
Denver Health Medical Plan is notifying 238,083 members that vendor SquareML leaked an access key on a public website. The key was online from 27 June 2025 until it was revoked on 18 August 2026 and gave read-only access to a database of names, dates of birth, health metrics, eligibility and admission data. DHMP found no evidence of misuse and reset passwords for all accounts with access.
Denver Health Medical PlanSquareMLHealth First Colorado+2
"When Algebraic Structure Meets Too Many Hints" is an ePrint paper (2026/2654) arguing that DDGR hint classes and too-many-hints thresholds are not standalone side-channel security metrics. In synthetic Kyber-512 experiments, NTT-domain residues recover the full secret with about half the hints the uniform mod-q bound predicts. Finite-precision FFT samples need 25% to 55% of the budget for uniform perfect hints. No new attack optimizations are proposed.
Seoul Metropolitan Police arrested seven people in October 2026 over a campaign that used generative AI to breach about 90 organizations and steal 8.2 million records. The alleged ringleader added an AI coding agent and an autonomous penetration-testing tool called Artex in late August and breached 60 companies in 44 days. Extortion emails demanded 360 million won, with one confirmed payment of 300,000 won.
ArtexClaude Code
Seoul Metropolitan Police AgencyWormGPTvibe hacking+2
Threat Intel, AI SecurityΒ· Published and added 26 Oct 2026
CyberForge CVE of the Day #087 covers CVE-2026-115889, a Cranelift value-index overflow in Wasmtime that can miscompile crafted WebAssembly and possibly escape the sandbox (CVSS 9.9, CWE-197 and CWE-843). Affected: below 36.0.19, and 37.0.0 up to but excluding 43.0.0; fixes are 36.0.19 and 43.0.0. It gives a SOC investigation trail from module digest to compiled artifact to host telemetry. EPSS is 0.454%, with no KEV entry.
The Policies of Programming Language Ecosystems is an October 2026 arXiv paper (cs.SE) that compares the policies of eight programming language ecosystems, including CRAN for R, across 38 policy categories. Three authors coded the policies with moderate agreement. Malware, intellectual property and content removal are covered everywhere, while package size limits, metric manipulation, typosquatting and binary uploads diverge.
CRAN
typosquatting
Supply Chain, GRCΒ· Added 28 Oct 2026, published 25 Oct 2026
Battledrones v14 adds "Compass Rose", a four-player map with four-way rotational symmetry, to Simon Willison's vibe-coded RTS game, dated 24 October 2026. A single prompt to Claude Opus 4.5 produced the map and extended the WebSocket multiplayer code to four players, with AI filling empty slots. Players join by share link or QR code. New teams are purple and green. The post gives no code or diff.
Socket's Threat Research team reports that malicious cache.yml workflows ran in the BerriAI/litellm repository on 10 and 12 October 2026, eleven days before the 21 October release attack that published litellm==1.107.0. Each commit pushed to a short-lived ci-tmp-<hex> branch, then deleted it, sending base64-encoded CI secrets to webhook.site. Evidence comes from Actions metrics and environment deployment records; 20 secret names were exposed.
LiteLLMGitHub ActionsGitHub+3
ci-tmp branchescache.ymllitellm==1.107.0+6
Supply Chain, DevSecOps, Blue Team & DetectionΒ· Published and added 23 Oct 2026
ADLC Security Best Practices is an Endor Labs guide to controlling what AI coding agents can access, run, install and ship. It covers inventorying agents and MCP servers, sandboxing, pre-execution hooks instead of AGENTS.md rules, install-time package checks against slopsquatting and malware, reachability-based triage of agent PRs, and CI/CODEOWNERS protections. A five-step worked example and coverage metrics are included.
AINews issue for 21-22 October 2026, built from 12 subreddits and 544 Twitter accounts. It leads with "Black October", a 48-hour run of AI-found vulnerability disclosures (PHP2Shell, a GitLab CVSS 10 flaw, a LiteLLM 1.107.0 password stealer, eMudhra CA keys). It also covers MiniMax M3.1 Flash, Carbon-X and Flint releases, Artificial Analysis coding-agent and CritPt v2 results, and a METR/Redwood audit of OpenAI monitoring.
vLLMLiteLLMPHP+14
Latent SpaceAINewsAnthropic+43
models, evals, ai-safetyΒ· Published and added 23 Oct 2026
Sentiment-Injected Sentence Embeddings (SISE) addresses a finding across eleven frozen text encoders (2021 to 2026): linear probes recover sentence polarity at up to 91.3% accuracy, but cosine similarity separates polarity by only 0.003-0.064. A small supervised-contrastive head (0.13M-0.69M parameters) trained on CPU raises retrieval MAP by up to 26 points, but the 128-d head loses 38-41 STS-B Spearman points. The paper also releases the Attribute Alignment Gap diagnostic.
Qwen3RAG
Qwen3-Embedding-4BJina-v5SISE+3
research, llm-appsΒ· Added 26 Oct 2026, published 23 Oct 2026
Sensorimotor Alignment (S^2) is a training-free offline metric that measures how well an embodied model's sensory representation aligns with the geometry of physical motion. It extends the Platonic Representation Hypothesis to motion trajectories. S^2 correlates positively with downstream robot task success across embodied model families, helps pick promising VLM backbones before fine-tuning into VLAs, and explains behaviours seen in VLAs and world action models.
vision-language modelvision-language-action modelworld action model
huggingface.copaperπ§ AI
