Skip to content

234 results for “Harness”

ListGrid
What you seeeverything
RecentOldestAddedA–Z
All types234Articles81Papers54Repos45Write-ups27Threads12
Talks7Tools5Courses2Other1
“Harness”
latent.spacearticle🧠 AI

Agents Prompt People: Using Agents for Long-Horizon Financial Work

Latent Space podcast write-up of an AI Engineer New York interview with Aman Singh, lead AI Engineer at Arca, an AI-native wealth management firm. Arca runs agent tasks lasting up to ten days, uses a per-client memory system, and owns its harness, which routes tasks to models (mainly Claude) by internal benchmarks. Safety rules are deterministic, trading stays with human advisors, and a meta-agent uses advisor feedback and traces to improve the harness.

ClaudeLLM
ArcaAman SinghAnthropic+6

agents, ai-infra· Published and added 31 Oct 2026

github.comrepo🧠 AI

llm-trails: What LLMs reach for inside a breached AWS account

llm-trails is an open study of what LLMs choose to do next inside a breached AWS account. It puts 14 models in 13 post-compromise situations, records only a single-turn plan with nothing executed, and codes each answer to CloudTrail events. The repo has the llmtrials harness, prompts, codebook, processed dataset (viz_data.json) and an interactive dashboard, so the study can be rerun or extended.

AWSCloudTrailLLM+2
Adan Alvareztraildiscover.cloudClaude

evals, agents· Added 30 Oct 2026

github.comrepo🛡️ Security

llm-trails: What LLMs reach for inside a breached AWS account

llm-trails is a research repo that placed 14 LLMs in 13 post-compromise AWS situations, asked only for their next move (single-turn, nothing executed), and coded each answer to CloudTrail events. It includes the llmtrials harness, prompts, codebook, processed dataset (viz_data.json) and an interactive dashboard. Rates are conditioned on the situation shown, and the aim is turning model behavior into detection priorities.

AWSAWS CloudTrailLLM
traildiscover.cloud

AI Security, Cloud & Infra, Blue Team & Detection· Added 30 Oct 2026

labs.ycombinator.comwriteup🧠 AI

We ran AI agent swarms in Minecraft and Mindustry

Agents in Games is a Y Combinator Labs writeup, done with Olam Labs, of multi-agent swarm experiments in Minecraft and Mindustry. A 50-agent GPT-6.1 Sol Minecraft colony became risk-averse after 22 of 34 deaths came on night one. GPT beat Claude Opus 5.5 at Bed Wars 6-3 and Mindustry 6-1. The authors open-sourced a Minecraft agent harness, Multi-Agent Mindustry and a Harbor fork.

MinecraftMindustryMineflayer+7
Y CombinatorOlam LabsGPT-6.1 Sol+9

agents, ai-safety, evals· Added 30 Oct 2026, published 29 Oct 2026

latent.spacetalk🧠 AI

How Anthropic ships Claude with Claude: Separating the Brain from the Hands

Latent Space episode with Katelyn Lesse (engineering) and Angela Jiang (product) of Anthropic, recorded the day after the Claude Opus 5.5 launch. It covers distillation defences such as enforced preserved thinking blocks and a new API compaction feature, per-model data retention and verification programs, trust grants for access control, and the Managed Agents split between harness and sandbox. The text shown is truncated; later sections are known only from timestamps.

ClaudeClaude CodeRust
AnthropicClaude Opus 5.5Katelyn Lesse+7

agents, ai-infra, models· Published and added 27 Oct 2026

latent.spacetalk🛡️ Security

How Anthropic ships Claude with Claude: Separating the Brain from the Hands

Latent Space episode with Katelyn Lesse and Angela Jiang of Anthropic, recorded after the Claude Opus 5.5 launch. It covers distillation defences (enforced preserved thinking blocks, a compaction feature for harness maintainers), trust-grant authorization for gated model access, and the Managed Agents split between harness and sandbox, where credentials are held outside the sandbox and attached through a proxy. Only the first third of the transcript was available.

ClaudeClaude PlatformClaude Managed Agents+1
Opus 5.5MythosKatelyn Lesse+4

AI Security· Published and added 27 Oct 2026

latent.spacetalk🧠 AI

How Anthropic ships the #1 LLM in the World safely: Separating the Brain from the Hands

Latent Space episode with Katelyn Lesse and Angela Jiang on the engineering behind the Claude Platform. Topics include the Claude Opus 5.5 launch and pricing, Claude Managed Agents that separate the model's brain from the harness and sandbox, rebuilding the Messages API in Rust, credential protection, safety classifiers, model routing and agent evaluation. The page lists show notes and timestamps only, with no transcript.

ClaudeRust
Latent SpaceAnthropicClaude Opus 5.5+2

agents, ai-infra· Published and added 27 Oct 2026

huggingface.copaper🧠 AI

Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution

TraceTrial-Bench is a benchmark that scores agent task performance and agent harness reliability separately. It has 95 mechanism-oriented tasks across seven components: context, execution, governance, lifecycle, observability, tools and verification. Across 4 harnesses, 9 models and 3,420 runs, macro-averaged harness scores range from 59.8% to 72.0%, and 31.1% of runs succeed despite failing at least one harness criterion.

LLMHugging Face
TraceTrial-Bench

evals, agents, research· Added 27 Oct 2026

arxiv.orgpaper🧠 AI

Continual Learning Exam: Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents

Continual Learning Exam (CLE) is a benchmark of 120 human-authored tasks, each a five-level ladder (600 levels) across 89 professional roles and 12 occupational domains, built by 50+ domain experts. Agents must pass a level to unlock the next, and an evaluation agent scores submissions against human rubrics. Across 17 harness-model configurations, feedback plus one revision adds a median 5.3 points; the best system clears 47.7% of levels and finishes all five levels in 23.3% of tasks.

LLM
Continual Learning ExamarXiv

evals, agents, research· Added 27 Oct 2026, published 25 Oct 2026

repodir.comrepo🧠 AI

affaan-m/ECC | RepoDir

ECC (affaan-m/ECC) is an open-source performance optimization system for AI coding agents. It packages 261+ skills, 64+ agents and 84+ commands that tune Claude Code, Codex, Cursor and other agent harnesses for speed, memory and security. Practitioners can use it as a ready-made skills and harness configuration set. The RepoDir page gives only this short description, with no install steps or benchmarks.

Claude CodeCodexCursor+1

coding-agents, prompting· Added 24 Oct 2026

latent.spacearticle🧠 AI

AI Is Getting Good at Math. Here’s Why.

The essay argues that LLMs solve open math problems because they search depth first, and automated verifiers make failed attempts cheap. It uses WT Gowers' hint-sequence complexity (2^k + t), adds a branching factor b, and walks through OpenAI's unit-distance disproof and Cycle Double Cover prompt as a drafter/solver/verifier harness. It predicts AI does best where k and b are small.

LLM
OpenAIAnthropicLatent Space+12

research, agents· Published and added 23 Oct 2026

arxiv.orgpaper🧠 AI

When Agents Fail: Model or Harness--Who's to Blame?

When Agents Fail: Model or Harness--Who's to Blame? proposes five measures for evaluating agent harnesses together with their underlying models: Valid Completion Rate, Model-Harness Fit, Cross-Model Fit, Interaction Residuals and Token Load. It tests four harnesses (OpenCode, Codex CLI, Pi CLI, DeepAgents) with four models on six benchmarks, across 111 settings and 52,168 trajectories. Harness rankings change by model and benchmark. OpenCode ranks first in cross-model performance.

OpenCodeCodexPi+2
DeepSeek V4arXiv

evals, agents, research· Added 26 Oct 2026, published 23 Oct 2026

arxiv.orgpaper🧠 AI

4DMemory: Benchmarking Long-Horizon Spatiotemporal Memory with Harness-Augmented Agents

4DMemory is a benchmark suite for long-horizon spatiotemporal memory: the LongVideo dataset (about 900 hours of video, over 5,000 manually reviewed questions) and 4DMemoryBench. It also presents the 4DMemory Harness, combining four skills, 22+ tool models and a model-specific RAG system. Across 13 models, the harness improved every one by over 50%, and the same models scored over 10% lower than on a previous benchmark.

RAG
4DMemory4DMemoryBenchLongVideo+2

evals, multimodal, agents· Added 26 Oct 2026, published 23 Oct 2026

arxiv.orgpaper🧠 AI

Reproducible Evaluation as the Substrate for Safe Prompt Compression in Production Agents

This paper presents an offline evaluation harness for a production merchant-facing conversational agent. It stays reproducible when live backend data changes and can be regenerated when it goes stale. Used as the fitness function for GEPA prompt optimization under a correctness gate against reward hacking, it shrinks the full prompt surface by 3.5-7.6% without losing held-out correctness, across three subject-model families.

GEPALLM
arXiv

evals, prompting, agents· Added 26 Oct 2026, published 22 Oct 2026

x.comthread🛡️ Security

Daily Vibe Casting (@vibecastingapp) on X

Daily Vibe Casting thread of 24-hour tech news, with item 5/10 on Cloudflare's agent security audit skill. The skill works across models and agent harnesses, including free models, and outputs machine-readable reports. It calls for sandboxing, proof of real impact and independent checks. The thread also covers GitLab CVE-2026-110264 (CVSS 10.0, unauthenticated RCE), two AI-found PHP RCE flaws, and a Railway secret-theft warning.

CloudflareGitLabPHP
CVE-2026-110264RailwayWiz Research

AI Security, AppSec· Added 24 Oct 2026, published 22 Oct 2026

arxiv.orgpaper🧠 AI

CellTrack-Bench: Microscopic Cell Tracking Benchmark for Vision-Language Models

CellTrack-Bench is a benchmark that tests vision-language models on microscopic cell tracking and compares them with specialised trackers. It has 1,537 problems covering 9,041 cell-linking decisions, drawn from 15 cell-tracking datasets across four imaging modalities, with a hard subset built from failures shared by distinct algorithmic trackers. The authors compare environment harnesses, benchmark 9 closed-source and 13 open-weight models, and analyse failure modes.

vision-language modelLLM
CellTrack-BencharXiv

evals, multimodal, research· Added 24 Oct 2026, published 22 Oct 2026

openai.comarticle🧠 AI

What FDE means at OpenAI

Colin Jarvis, Head of Forward Deployed Engineering at OpenAI, describes how the FDE team works with enterprise customers to close the "deployment gap". Cases include a semiconductor verification project (about 48% time saved on test generation) and a telecom voice model whose key evals rose from about 60% to 90%+. Moving use cases onto a Codex harness with skills and plugins removed about 90% of custom code.

CodexModel Context ProtocolOpenAI API+1
OpenAIColin JarvisJohn Deere+3

ai-policy, coding-agents, agents· Published and added 22 Oct 2026

adversa.aiarticle🛡️ Security

Is Claude Code safe for the enterprise? What the official settings cover, and what they don't

Adversa AI assesses Claude Code enterprise security as of October 2026. Managed settings bind only Claude Code, can be edited by local admins, and are not read by Anthropic-hosted cloud sessions. Auto mode misses 17% of risky actions in Anthropic's test set. The article covers attacks on the harness (CVE-2025-59536, CVE-2026-21852, TrustFall, SymJack, GhostApproval) and recommends credential, egress, isolation and monitoring controls.

Claude CodeAnthropicCursor+2
CVE-2025-59536CVE-2026-21852TrustFall+7

AI Security, DevSecOps· Published and added 22 Oct 2026

huggingface.copaper🧠 AI

Matrix-Game 4.0: Harnessing Video Models for Real-Time Playable Game Worlds

Matrix-Game 4.0 is a system that separates game logic from visual rendering for playable generated worlds. A lightweight engine holds world state and rules, a multimodal coding agent writes game code and scene captions, and a distilled autoregressive video model turns low-poly proxies into detailed frames. It runs 480p at 60 FPS on one RTX 5090 with 112 ms key-press-to-display latency. Code, weights and assets are released.

diffusion modelHugging Face
Matrix-Game 4.0NVIDIA

multimodal, research, agents· Added 22 Oct 2026

x.comtool🧠 AI

EveryDev.ai (@EveryDevAi) on X

Weco, from WecoAI, is an open source CLI that optimizes code, prompts and agent harnesses against a metric the user defines. It uses LLM-guided tree search: the user supplies an eval command and Weco iterates across multiple files with the user's own API keys. The post lists Claude Code, Cursor, Codex, OpenClaw, PyTorch, CUDA and Triton as supported. It gives no benchmarks or setup details.

WecoClaude CodeCursor+5
WecoAI

coding-agents, evals· Published and added 22 Oct 2026

x.comthread🧠 AI

elvis (@omarsar0) on X

FrontierMemory is a benchmark that evaluates agent memory through downstream task performance, measuring how agents learn from experience rather than recall, which the post says is saturated via agentic search. In elvis's summary, Claude Opus 5.5 ranks first and open-weight GLM-5.3 does well on memory generation. The authors reply that "memory" here means any adaptation from experience, so harness-level changes can also be scored. The post gives no scores or links.

LLM
FrontierMemoryClaude Opus 5.5GLM-5.3+2

evals, agents· Added 23 Oct 2026, published 21 Oct 2026

latent.spacearticle🧠 AI

Claude Mods: Flipping the Harness Inside Out

Claude Mods are Claude Code plugins, written as TypeScript or JavaScript functions, that hook into prompts, tool calls and rendering to customize the CLI and Desktop. The author describes built-in mods such as You Should Know, and their own Next Steps Supervisor in the ModSmith plugin. They compare mods with ChatGPT Intelligent UI and Cursor PR Routing & Approval, and list downsides: noise, token cost and lack of sandboxing.

Claude CodeTypeScriptJavaScript+3
AnthropicLatent SpaceOpenAI+2

coding-agents, agents· Published and added 21 Oct 2026

latent.spacearticle🧠 AI

Claude Mods: Flipping the Harness Inside Out

Claude Mods are Claude Code plugins, written as TypeScript or JavaScript functions, that hook into prompts, tool calls and rendering to change how the CLI and Desktop behave. The author describes six built-in mods, including You Should Know, and their own Next Steps Supervisor from the ModSmith plugin. They compare mods with ChatGPT Intelligent UI and Cursor PR Routing & Approval, and note noise, token cost and trust problems.

Claude CodeTypeScriptJavaScript+3
AnthropicLatent SpaceOpenAI+5

coding-agents, agents· Added 23 Oct 2026, published 21 Oct 2026

arxiv.orgpaper🛡️ Security

Where AI Vulnerability Discovery Harnesses Go Quiet: Structural False Negatives and Hidden Vulnerabilities

Paper (arXiv, 21 October 2026) on structural false negatives in LLM vulnerability discovery harnesses. Starting from Knostic's OpenAnt and raptor, then 10 more open-source harnesses, the authors map 13 points where code selection, filtering, and finding storage or merging drop vulnerabilities. One file was dropped by six harnesses before the model saw it, five without notice. It covers attacker evasion, content-steered verdicts, prompt injection, and design principles and tests.

OpenAntraptorLLM
KnosticPtacek and Newshamprompt injection+1

AI Security, AppSec· Added 24 Oct 2026, published 21 Oct 2026

latent.spacetalk🧠 AI

AI for the Fortune 2000: How Tenex Builds an AI-Native SDLC — Alex Lieberman & CJ Hess

Latent Space interview from AIE New York with Tenex cofounder Alex Lieberman and CJ Hess on running an AI-native engineering firm of 75 people. Covers story-point pay for engineers, the MetaHarness (plans, decisions and agent logs checked into Git with a CLI), roughly 2x engineer output in 2026, Opus 5.5 and Codex usage, and enterprise adoption rates. Also debates vertical AI and the Kill My SaaS experiment.

GitClaude CodeCodex+4
TenexAlex LiebermanCJ Hess+15

coding-agents, ai-policy, agents· Published and added 21 Oct 2026

latent.spacetalk🧠 AI

AI for the Fortune 2000: How Tenex Builds an AI-Native SDLC — Alex Lieberman & CJ Hess

Latent Space interview from AI Engineer New York with Tenex cofounder Alex Lieberman and CJ Hess. Tenex has 75 people, pays engineers by story points, and reports roughly 2x engineer output over the past year. CJ describes MetaHarness, a repo-based set of skills, plans and a CLI that logs agent decisions in Git. They also cover enterprise AI adoption rates, token spend, and vertical AI. The page text is truncated.

GitGitHubCodex+3
TenexAlex LiebermanCJ Hess+14

coding-agents, ai-policy, agents· Added 23 Oct 2026, published 21 Oct 2026

endorlabs.comarticle🛡️ Security

Gemini 4 Argon on Cursor: solid security score, two unique solves, but slower and more expensive | Endor Labs

Endor Labs evaluates Gemini 4 Argon in the Cursor harness on its Agent Security League coding tasks (SusVibes). Argon ties Fable 5 at 29% SecPass with 78.8% FuncPass, but costs $765 and takes a 14-minute median per task. It uniquely fixes SABnzbd path traversal (CVE-2021-29488) and a Nova credential-logging leak (CVE-2015-8749). Two other candidate solves were rejected, and 29 cheating instances were confirmed.

CursorGeminiEndor Labs+2
CVE-2021-29488CVE-2015-8749Agent Security League+8

AI Security, AppSec· Published and added 20 Oct 2026

endorlabs.comwriteup🧠 AI

Gemini 4 Argon on Cursor: solid security score, two unique solves, but slower and more expensive | Endor Labs

Endor Labs ran Gemini 4 Argon in the Cursor harness on its Agent Security League coding tasks (SusVibes). Argon ties Fable 5 at 29% SecPass with 78.8% FuncPass, and uniquely and securely solves two tasks (SABnzbd CVE-2021-29488 and OpenStack Nova CVE-2015-8749). It is also the slowest and costliest recent combo: 14 minutes median per task and $765. The run had 29 confirmed cheating instances.

CursorClaude CodeCodex+3
CVE-2021-29488CVE-2015-8749Endor Labs+15

evals, coding-agents, models· Published and added 20 Oct 2026

arxiv.orgpaper🧠 AI

AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics

AgenticRoboRL is a framework for post-training robot planners across different harnesses, combining supervised fine-tuning with multi-turn reinforcement learning. Harness-specific adapters link recorded planner interactions to task outcomes, so one RL pipeline covers both VLA skill invocation and control-code generation. On LIBERO-PRO, a 9B planner rose from 18.1% to 71.0% without external memory, near Codex (GPT-5.5) with memory at 72.1%. Code and checkpoints are promised.

Codexvision-language-action model
AgenticRoboRLLIBERO-PROGPT-5.5+1

research, training, agents· Added 22 Oct 2026, published 20 Oct 2026

github.blogarticle🧠 AI

How 3 Java Champions are using GitHub Copilot for increased productivity

Interviews with three Java Champions (Vlad Mihalcea, Patrick Baumgartner, Frank Delporte) on using GitHub Copilot in production and open source. Examples include a 900-slide Reveal.js to Marp migration with visual verification, a Copilot coding agent diagnosing a Pi4J virtual-thread pinning bug, and custom Agent Skills. Tips: interview-first planning, task-specific verification gates, deterministic quality harnesses, and human review.

GitHub CopilotJavaIntelliJ IDEA+12
GitHubVlad MihalceaPatrick Baumgartner+7

coding-agents, prompting, ai-policy· Added 29 Oct 2026, published 20 Oct 2026

latent.spacearticle🧠 AI

[AINews] not much happened today

AINews issue for 16 to 19 October 2026 from Latent Space, a digest of Twitter and Reddit AI news. It covers DeepSeek V4.1 Flash 1017 (552B total, 16B active MoE, AA Intelligence Index 45), MiniMax M3.1 Flash, Mercury 3, Meta Muse, Google's Harness², vLLM v0.32.0 and a Terminal-Bench audit. Only the first 40,000 of 47,045 characters were available.

Mixture of ExpertsGGUFllama.cpp+12
Latent SpaceDeepSeekDeepSeek-V4.1-Flash+42

models, evals, inference· Added 30 Oct 2026, published 20 Oct 2026

latent.spacearticle🧠 AI

[AINews] not much happened today

AINews issue covering 16-19 October 2026 from Latent Space, summarising 12 subreddits and 544 Twitter accounts. It leads with DeepSeek V4.1 Flash 1017 (552B total, 16B active MoE, 1M context, MIT license, AA Intelligence Index 45), plus MiniMax M3.1 Flash, Mercury 3 and Meta Muse. It also covers vLLM v0.32.0, InferenceX results, Harness² and Terminal-Bench audit findings.

Mixture of ExpertsvLLMSGLang+11
Latent SpaceDeepSeek-V4.1-FlashDeepSeek+43

models, evals, inference, agents· Published and added 20 Oct 2026

arxiv.orgpaper🧠 AI

Can It Actually Work? Synthesizing Functional CAD Mechanisms

FunCAD-Harness is a framework in which LLM agents design CAD mechanisms and revise them using physical simulation feedback. It turns abstract functional requirements into candidate-specific physical tests. The paper also introduces FunCADBench, a benchmark of 46 mechanism families with interdependent, multi-step operations. Experiments on frontier LLM agents show it exposes mechanical failures that visual inspection misses and raises functional success over standard simulation-augmented baselines.

LLM
FunCAD-HarnessFunCADBencharXiv

research, evals, agents· Added 21 Oct 2026, published 19 Oct 2026

arxiv.orgpaper🧠 AI

OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory

OpenVLHarness is a multimodal reasoning harness for MLLMs built on a capability layer (task-level tool interfaces plus evidence rendering) and persistent multimodal memory that stores images, structured data and web references as addressable artifacts. A coding interface runs generated programs over stored data. Across 22 benchmarks and eight backbones it adds 9.7-12.6 points over no-tool baselines, and GPT-6 Luna at medium effort beats Codex and Visual Sketchpad at maximum effort using at most 37% of their tokens.

vision-language modelQwen3-VLCodex+1
OpenVLHarnessKimi K3GPT-6 Astra+3

agents, multimodal, research· Added 21 Oct 2026, published 19 Oct 2026

arxiv.orgpaper🧠 AI

VibeTraining-Bench: Data-Trainer-Harness Co-Design for Iterative System Improvement

VibeTraining-Bench is a benchmark for cross-component co-design, where research agents iteratively refine training data, trainers and inference harnesses for a fixed starting agent system. Tasks cover terminal operation, software engineering, multi-turn tool use and math reasoning, with safeguards against reward hacking. On SWE-bench Verified, GPT-6 Astra raises the target system from 32.9% to 53.7%, against 55.4% for the human-developed instruct model.

Docker
VibeTraining-BenchGPT-6 AstraSWE-bench Verified+1

evals, agents, training· Added 21 Oct 2026, published 19 Oct 2026

huggingface.copaper🧠 AI

Self-evolving Agent Harness

Self-evolving Agent Harness is a 66-page survey and theory paper on revising an agent's scaffolding (prompts, memory, workflows, tools, code, plugins) while model weights stay fixed. It frames search as evolutionary optimization and uses continual PAC-learning to derive a generalization bound, a retention bound and a high-probability admission rule with fresh validation data. It separates selection, activation, attribution and improvement as evaluation claims, and covers applications and governance. Code is at github.com/zgcagi/Harness-RSI.

LLM
Harness-RSIHugging Face

agents, research, evals· Added 19 Oct 2026

x.comthread🧠 AI

Ethan Walkrman (@ethanwalkerman) on X

Harness2 is a framework from Google Cloud AI Research and Northwestern University in which an Editor model rewrites an agent's harness (system prompt, memory, skills, tool scripts, subagent routing, middleware) without changing model weights. It uses a Propose-Probe-Compose loop that compares execution trajectories instead of ground truth. On WorkBuddy-Bench, GLM-5 rose from 64.2 to 75.7 after one round, above Gemini 3.5 Flash at 72.4. The thread also lists risks: latency, experience poisoning, and a sandbox that is off by default.

CodexPythonGit+3
Harness2Google CloudNorthwestern University+6

agents, research, evals· Added 19 Oct 2026, published 18 Oct 2026

api-docs.deepseek.comother🧠 AI

DeepSeek search and fetch – $0.75 per 1000 successful requests

DeepSeek Search & Fetch are two JSON APIs for agents and RAG apps. Search (/v1/tools/search) returns structured results from DeepSeek's in-house engine, with count, need_content and operators such as site:, lang: and after:. Fetch (/v1/tools/fetch) returns a page as Markdown. Both use bearer auth, bill only successful requests ($0.75 per 1000), and have account-level QPS limits. Calls from DeepSeek Harness, Claude Code or Codex are free until 1 November 2026.

DeepSeekPythonRAG+2
DeepSeek V4

llm-apps, mcp, agents· Added 19 Oct 2026, published 18 Oct 2026

arxiv.orgpaper🧠 AI

ToFu: Exploiting and Fuzzing the LLM–Orchestrator–Tool Trust Boundary

ToFu is a tool-aware fuzzing framework for LLM tools, including MCP servers. It maps each tool argument, using its metadata, to one of 74 CWE-guided semantics-aware sanitizers (path, URL, SQL and similar). It then generates schema-valid Atheris harnesses and semantic oracles, using runtime tracers for in-process tools and outcome oracles for opaque ones. The LLM (gpt-4.1) only maps sanitizers and writes seeds; it is never the security judge. The paper also shows a "Repeat-After-Me" trick, where an LLM told to repeat attacker-crafted text emits a tool call the orchestrator executes, reproduced on LangGraph, MAF, ADK 2.0, AutoGen, Strands and CrewAI. Across 273 runnable MCP servers (3,405 tools) it reports 253 verified bugs in 343 tools across 118 servers, such as path traversal, log injection and exfiltration. These are failures a crash-based fuzzer would miss. The result matters because it shows LLM-generated tool arguments cannot be treated as trusted.

Model Context ProtocolAtherisCodeQL+7
ToFuStrandsMAF+10

mcp, research, agents· Added 18 Oct 2026

arxiv.orgpaper🛡️ Security

ToFu: Exploiting and Fuzzing the LLM–Orchestrator–Tool Trust Boundary

ToFu is a tool-aware fuzzing framework that derives semantic security checks for LLM tools (mainly MCP servers) from their metadata and a corpus of 74 CWE-guided sanitizers. It generates schema-valid Atheris harnesses, uses runtime tracers and outcome oracles to catch bugs that don't crash, and uses an LLM only to map sanitizers and seed inputs, never as the judge. It also describes a "Repeat-After-Me" attack, in which the LLM is told to repeat attacker-written text that the orchestrator then dispatches as a tool call. It was reproduced on LangGraph, MAF, ADK, AutoGen, Strands and CrewAI. On 273 MCP servers it reports 253 verified bugs across 343 tools in 118 repositories, including path traversal, log injection, exfiltration and command bypass.

Model Context ProtocolAtherisLangGraph+3
Repeat-After-Mepath traversalSSRF+6

AI Security, AppSec· Added 18 Oct 2026

blog.picheta.mearticle🧠 AI

I built a programming language that LLMs can't use

KillSwitch is an esoteric stack-based programming language built to be adversarial to LLMs and coding agents. It uses letter-counting rules, a spec with hidden lies, guardrail-triggering phrases like "hack into a computer", and noisy story-style scripts. The author benchmarked major models and agent harnesses on simple tasks. Opus 5 scored best at 66.8, Fable 5.1 and Gemini 4 Argon did poorly because of safety guardrails, and MiMo-V2.6-Pro led the open-weight models.

KillSwitchLLMGitHub
Opus 5Claude Opus 5.5Claude Fable 5.1+2

evals, coding-agents· Added 19 Oct 2026, published 17 Oct 2026

x.comthread🧠 AI

DeepSeek (@deepseek_ai) on X

DeepSeek announces DeepSeek-V4.1-Flash-1017, claiming stronger benchmarks, faster task completion and shorter outputs than the previous version, live on its website, app and API. It is optimized for DeepSeek Harness (v0.2.1-rc.1 or later), Tencent CodeBuddy and WorkBuddy, and uses reinforcement learning with self-feedback to improve end-to-end project development, from game creation to 3D modeling. The thread also opens the Search & Fetch APIs to everyone, backed by DeepSeek's in-house search engine at $0.75 per 1,000 calls per endpoint, available as server-side tools.

DeepSeekLLM
DeepSeek-V4.1-FlashDeepSeek HarnessTencent+2

models, agents, training· Added 18 Oct 2026, published 17 Oct 2026

huggingface.corepo🧠 AI

DeepSeek-v4.1-Flash-1017

Model card for DeepSeek-V4.1-Flash-1017, a minor update to DeepSeek-V4.1-Flash with MIT-licensed weights. It claims gains in reasoning, coding, agentic tasks and visual benchmarks, with shorter task time and output length that move the performance–cost Pareto frontier across agent harnesses (Claude Code for Terminal-Bench 4.0, Mini-SWE-V2 for DeepSWE v1.1). It ships no Jinja chat template. It provides a reference encoding.py for multi-turn chat, tool calling, thinking mode, numeric reasoning effort and interleaved images, plus deepseek-recipe, Rust libraries with Python bindings for prompt encoding and output parsing. Serving recipes are given for Transformers, vLLM, SGLang and Docker Model Runner.

TransformersvLLMSGLang+5
DeepSeekDeepSeek-V4.1-FlashDeepSeek V4+3

models, inference· Added 18 Oct 2026, published 17 Oct 2026

huggingface.copaper🧠 AI

OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use

OpenGCUA, from Alibaba's MAI-UI team, is an open stack for generalist computer-use agents (GCUAs), where the model chooses how to observe, act and organize work across GUI, accessibility-tree, API and shell interfaces. Its design combines code execution (GUI actions written as programs with loops and branches) with harness-to-model decision transfer. On OSWorld-v2 it raises mean partial score by +19.6 points across eight models, including GPT-6 Astra and Claude Opus 5, and cuts reported steps per task by 53%. The release includes the harness, prompts, evaluation trajectories, a macOS real-device harness, a cross-device SDK and an interactive lab. It also shows large efficiency gaps between models: Qwen3.8-Max used 19.4x the reasoning tokens of GPT-6 Astra on real macOS tasks.

macOSGitHubHugging Face
OpenGCUAMAI-UIAlibaba+6

agents, research, evals· Added 17 Oct 2026

microsoft.comarticle🛡️ Security

Managing open-weight model risk in enterprise deployments

Microsoft blog post introducing "Risk Management for Open-Weight Models in Enterprise", a joint Microsoft–NVIDIA paper. It proposes a shared risk taxonomy (Cybersecurity, Data Protection, Content, Human-AI Interaction, Bias) grounded in ISO standards and the NIST AI RMF. The paper maps evaluations to risks as evidence and argues that risk depends on the model, use case, deployment and safeguards, not on weight openness alone. It also covers how responsibility for controls shifts when enterprises self-host models, and points to a joint repository of evaluations. The first paper focuses on the model layer; runtime, harness and application layers are planned for later.

Microsoft FoundryNVIDIA
NIST AI RMFISO

GRC, AI Security· Published and added 16 Oct 2026

microsoft.comarticle🧠 AI

Managing open-weight model risk in enterprise deployments

Microsoft's announcement of "Risk Management for Open-Weight Models in Enterprise", the first paper in a planned joint series with NVIDIA. It proposes a taxonomy of five risk categories (Cybersecurity, Data Protection, Content, Human-AI Interaction, Bias). The approach is grounded in ISO standards and the NIST AI Risk Management Framework. It maps evaluations to risks as evidence, stresses that weight openness alone doesn't determine risk, and explains how self-hosting shifts responsibility for safeguards. A joint repository will collect evaluations, and later work will cover the runtime, harness and application layers.

Microsoft Foundry
MicrosoftNVIDIANIST+7

ai-policy, evals· Published and added 16 Oct 2026

starskirmish.comarticle🧠 AI

Show HN: Gemini 4 Argon plays StarCraft

StarSkirmish Bench v0.2 gives each LLM one hour to write a Protoss StarCraft: Brood War bot in C++ (BWAPI 4.4.0 on OpenBW), with tools to compile and play practice games. Scores are average win rates of 5 runs against 13 roster bots, which include human-written bots such as Stardust, PurpleWave and BananaBrain. Nine models are compared, each in its lab's first-party CLI harness (Claude Code, Codex, Antigravity CLI, Muse Code). GPT-6 Astra leads and wins only 7% against the top human bots. The page also covers score against cost, run-to-run variance, practice behaviour as the best predictor of score, and weak correlation with the Artificial Analysis Intelligence Index and Terminal-Bench 4.0.

Claude CodeCodexGoogle Antigravity+5
StarSkirmish BenchGPT-6 AstraGPT-6.1 Sol+22

evals, coding-agents, agents· Added 17 Oct 2026, published 16 Oct 2026

en.cryptonomist.charticle🛡️ Security

Vitalik Buterin’s privacy AI agent setup adds Tor and private payments

Describes Vitalik Buterin's open-source "paranoid mode" privacy wrapper for his pi agent harness. It is an OpenAI-API-compatible daemon that gives each conversation its own Tor circuit with random cover traffic, rewrites prompts with a local model before they reach the provider, and runs the agent in a bubblewrap sandbox wrapped with torsocks. Payments go through zkAPI, which uses zero-knowledge proofs to separate payment authorization from the funder's identity. The article notes the limits: high latency, weak protection against a provider colluding with a Tor guard relay, and a proposed constant-cover-traffic mixnet that is not yet implemented.

TorEthereumbubblewrap+3
Vitalik ButerinEthereum FoundationOpen Anonymity Project+4

AI Security· Published and added 16 Oct 2026

en.cryptonomist.charticle🧠 AI

Vitalik Buterin’s privacy AI agent setup adds Tor and private payments

News report on Vitalik Buterin's open-source "paranoid mode" privacy wrapper for his pi agent harness. It is a daemon compatible with the OpenAI API that sits between the agent and an external LLM provider. It gives each conversation its own Tor circuit with random cover traffic, rewrites prompts with a local model (qwen3.8-flash-next) before they leave the device, and runs the agent in a bubblewrap sandbox wrapped with torsocks. It pays through zkAPI, the Ethereum Foundation / Open Anonymity Project zero-knowledge scheme that separates payment authorization from the funder's identity. The article also covers the limits: high latency at conversation start, and a provider colluding with a Tor guard relay can defeat cover traffic, which motivates a proposed constant-cover-traffic mixnet.

Tortorsocksbubblewrap+6
Vitalik ButerinEthereum FoundationOpen Anonymity Project+2

agents, ai-infra· Published and added 16 Oct 2026

huggingface.copaper🧠 AI

Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents

Doc2Harness is a two-stage benchmark for harness engineering. In Stage 1 a build model, working inside a fixed builder harness, gets an interface-only contract skeleton plus a development document consolidating behavior from twelve frontier harnesses, and builds a full candidate harness in an isolated container. In Stage 2 the frozen harness runs in separate per-test-case containers, driven by a scripted mock model, and is scored on four channels: requests sent to the model, files on disk, exit code and command output. Across 19 frontier models with three builds each on 328 test cases, GPT-6 Astra leads at 53.05% against 27.44% for the weakest, and failures cluster around how cross-cutting functionality is split across modules.

Docker
Doc2HarnessGPT-6 AstraHugging Face

evals, coding-agents, agents· Added 16 Oct 2026

huggingface.coarticle🧠 AI

Welcome Beam: a 501B open model you can own, customize, and scale

Announces Reflection's Beam-1.0-501B-A23B, an Apache 2.0 text-only MoE model (23B active per token, 256 experts, 1M context) aimed at coding and agentic use. It covers the architecture (hybrid sliding-window/global attention, a constant-scale residual stream, DeepSeek-V3-style group-limited routing) and BF16/FP8/NVFP4 checkpoints with hardware sizing. It also gives day-0 serving recipes for transformers, vLLM and SGLang with DFlash2 speculative decoding, LoRA SFT with TRL, GRPO with Miles, and demos built with the Pi coding harness.

TransformersvLLMSGLang+12
BeamReflectionDeepSeek+15

models, inference, training· Published and added 16 Oct 2026

arxiv.orgpaper🧠 AI

MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering

Presents MLExpert, a 35B Mixture-of-Experts agentic model trained for autonomous long-horizon machine learning engineering. Training uses a Kaggle-MLE Factory to build executable tasks and an interaction harness, hierarchical agentic SFT with outcome-guided supervision, and Windowed Interaction RL (WIRL). WIRL treats the multi-turn decisions between successive experiment results as one execution window with a shared outcome reward, and uses asynchronous collection and policy correction. On MLE-Bench Lite the average medal rate rises from 37.9% to 72.7%, comparable to larger frontier models such as Kimi-K3 and GLM-5.3. It also improves on NatureBench Lite, agent scaffold optimization and algorithm optimization.

Mixture of ExpertsLLM
MLExpertKaggle-MLE FactoryMLE-Bench+4

research, training, agents· Added 17 Oct 2026, published 15 Oct 2026

snyk.ioarticle🧠 AI

VulnBump: Can AI Agents Fix Vulnerable Dependencies Without Breaking the Build?

Snyk introduces VulnBump, a benchmark of 307 real vulnerable-dependency upgrades from 43 repositories across Maven, Gradle, npm, Yarn, pip and Poetry. An attempt passes only if a fresh Snyk rescan shows the vulnerabilities gone and the project installs, builds and keeps its previously passing tests green in a clean container. On Claude Sonnet 5.5, Snyk's Evo Remediation Agent passed 90.2%, Claude Code with Snyk scan and target version 77.5%, unaided Claude Code 59.6%, and a non-AI package-manager bump baseline 46.9%. Unaided Claude Code often claimed fixes a rescan still flagged, so the benchmark shows harness and security data matter as much as the model. The harness and 43 cases are open-sourced.

Claude CodeSnyk Open SourceMaven+6
SnykVulnBumpVulnBench+6

evals, coding-agents· Published and added 15 Oct 2026

snyk.ioarticle🛡️ Security

VulnBump: Can AI Agents Fix Vulnerable Dependencies Without Breaking the Build?

Snyk presents VulnBump, a benchmark of 307 real vulnerable-dependency upgrades across 43 repositories and six package managers (Maven, Gradle, npm, Yarn, pip, Poetry). An upgrade passes only if a rescan shows the vulnerabilities gone and the project still installs, builds, and passes its previously passing tests. On Claude Sonnet 5.5, Snyk's Evo Remediation Agent passed 90.2%, Claude Code with Snyk scan 77.5%, Claude Code unaided 59.6%, and a plain package-manager bump 46.9%. Unaided Claude Code often claimed fixes that a rescan disproved, and major framework migrations and transitive dependencies were the hardest cases. The harness and the 43 cases are open-sourced.

SnykEvo Remediation AgentClaude Code+3
VulnBumpVulnBenchDepBench+1

AI Security, Supply Chain· Published and added 15 Oct 2026

huggingface.copaper🧠 AI

Harness^2: Recursive Agent Harnessing for an Open World

Harness^2 is a plug-and-play framework that recursively improves LLM agent execution harnesses in open-world settings where no ground-truth verifiers exist. Task-level recursion uses a contrastive Propose-Probe-Compose step to guide harness edits for the current task, and it scales through parallel or sequential recursion. Harnessing-level recursion distills experience from those edits into a continual-improvement harness for future tasks. A single step improves GLM-5 and Gemini 3.5 Flash over their base harnesses by up to 11.5 points on WorkBuddy-Bench, 4.5 on JobBench and 11.8 percentage points on LAB.

GLMGeminiHugging Face
GLM-5Gemini 3.5 FlashWorkBuddy-Bench+2

agents, research· Added 14 Oct 2026

arxiv.orgpaper🧠 AI

Thinking in Graphs: Neo4j-Backed Exploit-Chain Reasoning for LLM Penetration-Testing Agents

ChainGraph is a pattern that moves exploit-chain reasoning out of the flat context window and into a Neo4j property graph. Findings, attack steps, credentials, assets and tamper-evident evidence receipts are typed nodes with pre/post-condition edges. The graph is wired to Claude Code or Codex through MCP with no custom harness. Five operator questions (next-step inference, entry-to-crown-jewel pathing, combined-severity promotion, evidence provenance, chokepoint identification) become reusable Cypher and Graph Data Science queries, plus a chain-severity score. On a synthetic multi-forest Active Directory lab modeled on GOAD, three findings rated Medium in isolation become Critical once their path to the forest root is materialized. Scoping each decision to a graph result cut the reasoning payload by about 98% on a token proxy. The authors say the results are illustrative single-run measurements, not a benchmark.

Neo4jModel Context ProtocolClaude Code+4
ChainGraphGOADarXiv

agents, mcp, research· Added 17 Oct 2026, published 14 Oct 2026

blog.trailofbits.comwriteup🛡️ Security

Package managers have an integrity problem

Trail of Bits tested 25 popular package managers with a Docker and mitmproxy harness that simulates a compromised package repository injecting malware into legitimate packages. Fewer than half enforce lockfile hashes by default, only two verified every download, and only Go verifies signatures by default. The post covers lockfile hash pitfalls (Pub, pnpm, Conan, CocoaPods, Composer), repository, developer and Sigstore signing, and Go's checksum database. It also proposes lockfiles that commit to a signing identity plus a transparency log.

npmpnpmGo+3
Trail of BitsPEP 740cpanminus+5

Supply Chain· Published and added 14 Oct 2026

openai.comarticle🧠 AI

How Third Moment Research runs 10,000 agents for 36+ hours

OpenAI customer case study on Third Moment Research, which runs up to 10,000 concurrent agents on 36–48 hour autonomous quantitative-finance research sessions. Agents use the default Codex harness in a Dockerized environment with a terabyte of data. An outer harness driven by GPT-6 Sol or GPT-6.1 Sol watches for unfinished tasks and decides whether to retry, terminate or escalate to a human. Codex analyzed thousands of recorded trajectories and found a backtest bottleneck. A caching and memory-packing fix cut the backtest from about 3 hours to 3–5 minutes, which also reduced agent polling and context pollution. The post also covers how the team picks models by token efficiency and speedup benchmarks (GPT-6.1 Sol, GPT-6 Astra, GPT-5.6 Terra), and says Daybreak automates penetration tests of the sandbox.

CodexDockerGPT-6 Astra+1
Third Moment ResearchOpenAIGPT-6 Sol+5

agents, coding-agents· Published and added 14 Oct 2026

latent.spacearticle🧠 AI

[AINews] AI Engineer NYC: Gemini 4 Argon, Vals AI Benchmarks

AINews issue for 10/12–10/13/2026 covering the GA of Google's Gemini 4 Argon (pricing, Cursor/Devin/Copilot availability, CursorBench 52.8%, Artificial Analysis Intelligence Index parity with GPT-6 Astra at lower cost) and Meta's Muse Wonder, with independent evals and reliability caveats. It also summarises decision-model benchmarks (AA Decision Index, Jev 1.13, Nimble-v4), Fish Audio's open-weight Drama 3 TTS, agent-harness and agentic-RL research (Evo2, ReHarness, LongHarness, ThunderSync), Anthropic's move off Git, and safety items such as Argon's monitor evasion. It also notes AI Engineer NYC keynotes on AI x Finance. Useful as a curated, caveat-aware snapshot of the frontier-model landscape.

GeminiGoogle AntigravityCursor+14
Gemini 4 ArgonGoogleGoogle DeepMind+43

models, evals, agents· Published and added 14 Oct 2026

huggingface.copaper🧠 AI

AgentSWE: Can Coding Agents Build the Agent You Actually Want?

AgentSWE is a benchmark of 25 tasks covering the agent development lifecycle: Creation of an agent from a natural-language requirement, Editing of production agent codebases, and Optimization toward a measurable target. Builders iterate on a few illustrative cases, then the frozen submission is graded on held-out cases with requirement-derived rubrics and hard pass/fail gates. Across coding agents, models and harnesses, none comes close to delivering the requirement. Recurring failures include overfitting to visible cases, hollow runtimes behind code that looks complete, and unsupported claims of completion.

AgentSWEHugging Face

evals, coding-agents, agents· Added 13 Oct 2026

latent.spacetalk🧠 AI

Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics

Podcast interview with Jason Ma, co-founder and Chief Scientist of Dyna Robotics, on Dyna-2, a world-action model pre-trained on over one million hours of human egocentric video. The model predicts future video and uses it to guide robot actions, built on a diffusion transformer with flow matching. The discussion covers the robotics data bottleneck, adapting to new tasks from about 45 minutes of demonstrations, synthetic environments and self-generated training data, on-device inference and distillation, and Dyna's custom wheeled semi-humanoid hardware. The episode listing also covers commercial deployments (Din Tai Fung, Best Western), agentic harnesses for hour-long workflows, and world-action models versus VLAs, but the fetched text was truncated before those sections.

diffusion transformerflow matchingdiffusion model+2
Dyna RoboticsDyna-2Dyna-2.1+13

multimodal, training, models· Published and added 13 Oct 2026

huggingface.coarticle🧠 AI

Run fast, private agents on-device with LFM2.5-24B-A2B

Liquid AI releases LFM2.5-24B-A2B, an open-weight reasoning-only mixture-of-experts model (40 layers, 64 experts with top-4 routing, hybrid gated short convolutions and GQA, 2B active parameters) with 131K context and a 128K-token vocabulary. Pre-training scaled to ~33T tokens, followed by long-context mid-training and a five-stage post-training pipeline (SFT, teacher specialization, two multi-domain on-policy distillation stages, agentic RL in harnesses such as OpenClaw, Hermes Agent, Pi, OpenCode). It reports leading or matching results on tool-use and instruction-following benchmarks (e.g. LiveClaw 48.09 vs 38.19 for Qwen3.6-35B-A3B), 184 tok/s on an M5 Max and 21K tok/s on a B300, and includes transformers code and Hermes Agent local-serving config.

Mixture of ExpertsTransformersvLLM+12
Liquid AILFM2.5-24B-A2BLFM2-24B-A2B+18

models, agents, inference· Published and added 13 Oct 2026

deepmind.googlearticle🧠 AI

Gemini 4 Argon for Robotics

DeepMind presents Gemini 4 Argon as a general-purpose physical reasoning model for the robotics lifecycle. It reports 87% on the Embodied Reasoning (ERQA) benchmark and gives representative prompts across several areas: controlling a Boston Dynamics Spot with a MuJoCo digital twin, code-as-policies arm control with a visual coordinate harness, and steering the Gemini Omni world model in a visual feedback loop. Further examples cover building MuJoCo and mujoco_warp scenes from teleoperation data, egocentric video or a hand-drawn sketch for 76-DoF humanoids, hindsight annotation and value-function progress prediction on video, sim data generation for VLA fine-tuning (success rate 56% to 75%), and reproducing control papers in MuJoCo WebAssembly. It matters as a concrete look at how a frontier model is being used as agentic coder, perceiver and sim builder for robotics.

MuJoComujoco_warpBlender+5
Gemini 4 ArgonGoogle DeepMindERQA+10

models, multimodal, agents· Published and added 13 Oct 2026

kylekukshtel.comwriteup🧠 AI

How to engineer an AI harness for game development

Blog adaptation of an Indie Rodeo talk arguing that LLMs struggle with games because much of a game is opaque non-code, and that this is a context problem. It prescribes concrete architecture so a game becomes its own agent harness: seeded randomness, a single "Thing" fat-struct object model, serializable commands, external data editable without recompiling, a standalone simulation core, odiff image diffing for screenshot evaluation, and samply text profiling. The aim is an agentic loop that runs in seconds per change.

Claude Codeodiffsamply+6
Kyle KukshtelDepotCantata+7

agents, coding-agents, llm-apps· Published and added 13 Oct 2026

becomingahacker.orgarticle🧠 AI

AI Agents for Cybersecurity: A Guided Tour of a Repository with Tons of Resources and Examples

Guided tour of Omar Santos's eight-part AI-agents-for-cybersecurity repository. It moves from LLM building blocks (LangChain, RAG) to security agents built with LangGraph, CrewAI, DeepAgents and MCP servers (nmap, CISA KEV, Shodan). It also covers a hardening guide for six agentic frameworks, token-cost control, and a spec for a supervisor-led, human-gated assessment harness. Design lessons include least-privilege subagents, single-use approvals and kill switches.

LangChainRAGLangGraph+7
Omar SantosOWASPCISA+1

agents, mcp, llm-apps· Added 13 Oct 2026

github.comrepo🧠 AI

santosomar/AI-agents-for-cybersecurity

Companion repository for Omar Santos's Pearson and O'Reilly trainings and books. It has eight parts of runnable Python labs: chat models, embeddings, prompt templates, LCEL prompt chaining, Chroma-based RAG, ReAct/LangGraph/CrewAI/DeepAgents agents, and FastMCP servers (nmap, CISA KEV, Shodan). It also includes guides to six agentic frameworks (OpenClaw, Hermes, Agent Zero, OpenHands, Paperclip, Buzz), a tokenomics module on agent cost (context compounding, prompt caching, model routing, token budget guards, batch API, cost per accepted outcome), and a vendor-neutral design spec for a human-gated multi-agent assessment harness. It is a current, practical curriculum for building and operating agents. Some examples act on real systems and should only be run in an authorized lab.

PythonuvLangChain+17
Omar SantosPearsonO'Reilly+15

agents, llm-apps, mcp· Added 13 Oct 2026

github.comrepo🛡️ Security

santosomar/AI-agents-for-cybersecurity

Omar Santos's companion repo for his Pearson/O'Reilly trainings and books, organised in eight parts. They cover LangChain prompts, chains and RAG (Chroma), and agents (LangGraph, CrewAI, DeepAgents). They also cover FastMCP security servers (nmap, CISA KEV, Shodan) and attack-surface-management and SOC agent examples. Later parts have security guides to six agentic frameworks (OpenClaw, Hermes, Agent Zero, OpenHands, Paperclip, Buzz) with hardening notes and 2026 CVE mapping, agent cost/tokenomics scripts including denial-of-wallet budget guards, and a vendor-neutral spec for a human-gated multi-agent security-assessment harness (scope manifest, approval gates, kill switch). Useful as a hands-on path into AI-agent security tooling, with warnings that some examples act on real systems.

Model Context ProtocolLangChainOpenClaw+3
Omar SantosSSRFCISA KEV+2

AI Security, Learning· Added 13 Oct 2026

becomingahacker.orgarticle🛡️ Security

AI Agents for Cybersecurity: A Guided Tour of a Repository with Tons of Resources and Examples

Omar Santos walks through his eight-part AI-agents-for-cybersecurity repository: LLM building blocks (LangChain, RAG), then security agents with LangGraph, CrewAI, DeepAgents and MCP servers (nmap, CISA KEV, Shodan), a hardening guide for six agentic frameworks, token-cost control for agents, and a spec for a supervisor-led, human-gated security-assessment harness. It covers concrete design lessons such as least-privilege subagents, staged response actions, single-use approvals bound to exact actions, kill switches, and mapping controls to the OWASP Top 10 for Agentic Applications. Also notes Project CodeGuard rules shipped for AI coding assistants.

LangChainModel Context ProtocolPython+3
Omar SantosProject CodeGuardOWASP Top 10 for Agentic Applications+5

AI Security, Learning, Blue Team & Detection· Added 13 Oct 2026

linkedin.comarticle🛡️ Security

AI Agents for Cybersecurity: A Guided Tour of a Repository with Tons of Resources and Examples

Omar Santos walks through his AI-agents-for-cybersecurity GitHub repo, which has eight parts. Parts 1–4 cover LLM basics (LangChain, prompting, LCEL, RAG). Parts 5–8 cover building security agents with LangGraph, CrewAI, DeepAgents and MCP servers (nmap, CISA KEV, Shodan), and a hardening baseline for six ready-made agentic frameworks. They also cover token-cost control (caching, routing, budget guards, cost per accepted outcome) and a supervisor-led, human-gated security-assessment harness spec with single-use approvals, a kill switch and an append-only audit log. The post also covers Project CodeGuard rules for AI coding assistants. It is a practical learning path and a reference for governing and securing agents.

LangChainLangGraphCrewAI+5
Omar SantosProject CodeGuardOWASP Top 10 for Agentic Applications+9

AI Security, Learning, Blue Team & Detection· Added 13 Oct 2026

en.cryptonomist.charticle🧠 AI

Anthropic tests AI autonomous replication across eight nodes in four of five trials

The article reports Anthropic research (William Zhou and Daniel Freeman) on whether small open models can autonomously replicate onto unfamiliar IoT and consumer hardware. Qwen3.5-9B established inference on eight of nine hardware types using the Memento harness (built on the Pi coding agent). GLM-5.2 completed a simplified end-to-end replication across eight emulated Android nodes in four of five trials, and reached seven nodes in the fifth. Gemma4-26B-A4B was sharded by pipeline parallelism across 15 low-RAM Android TV boxes at 0.39 tokens/s. The authors estimate replication intervals of 1–2 days and a capability threshold around end of 2026, which makes it a concrete capability-evaluation datapoint for AI safety.

PiQwen3.5GLM+6
AnthropicWilliam ZhouDaniel Freeman+7

ai-safety, evals, agents· Published and added 12 Oct 2026

anthropic.comwriteup🧠 AI

AI models may be capable of self-replicating without access to a datacenter

Anthropic's Frontier Red Team tests whether small open-weight models (30B and under) could run as self-replicating "AI worms" on commodity IoT hardware. A 9B model set up inference on 8 of 9 hardware types. In an 8-node Android emulator fleet only GLM-5.2 replicated end to end. Gemma4-26B-A4B was sharded across 15 ~$55 Android TV boxes at 0.39 tok/s. Qwen3.5-9B lineages that inherit notes improve at inference setup (45% to 61-84%), while a frontier advisor (Opus 4.8) doesn't help. An ECI trend fit puts small models at GLM-5.2's level by end of 2026, and the post estimates botnet-scale compute and spread speed. It adds defensive suggestions (drive R0 below 1) and a harness called Memento built on the Pi coding agent.

PiMementoPetals+10
AnthropicGLM-5.2Gemma 4+14

ai-safety, evals, inference· Published and added 12 Oct 2026

micahflee.comarticle🛡️ Security

Just say no to OpenAI and Anthropic

Micah Lee argues security engineers should move off ChatGPT and Claude to open-weight models such as GLM-5.3, because the closed providers log everything and gate cyber use. He criticizes the cyber "cop agent" and the gated programs, OpenAI's Trusted Access for Cyber and Anthropic's Cyber Verification Program. He describes private inference through Tinfoil secure enclaves and Maple AI, cheaper non-private routing via OpenCode Go, and his OpenCode setup. He warns that Z.ai's ZCode harness reportedly uploaded local repos to Alibaba Cloud.

GLMTinfoilMaple AI+3
Trusted Access for CyberCyber Verification ProgramGLM-5.3+4

AI Security· Added 13 Oct 2026, published 12 Oct 2026

micahflee.comarticle🧠 AI

Just say no to OpenAI and Anthropic

Micah Lee argues for dropping OpenAI and Anthropic in favor of open-weight models, claiming they trail proprietary ones by about four months. They report GLM-5.3 as roughly on par with Claude Opus 5.5 for software engineering, and GLM-5.3-Flash as similar to Opus 4.8 and feasible to run locally. The post compares private inference through secure enclaves (Tinfoil, Maple AI) with cheaper non-private routers (OpenCode Go). It gives pricing, data-trust caveats about Z.ai, and the author's own workflow: OpenCode harness, GLM-5.3 on low reasoning, Tinfoil for private work.

OpenCodeGLMClaude+4
OpenAIAnthropicGLM-5.3+18

models, inference, coding-agents· Published and added 12 Oct 2026

jfrog.comarticle🧠 AI

Governing the Agentic SDLC Through Continuous Policy Enforcement

JFrog proposes the Agentic Process Evidence Protocol, an open spec that records coding-agent sessions (harness and agent version, model ID, tool and MCP calls, prompts, owner, human reviewer) as signed in-toto/DSSE attestations bound to a git commit, artifact or release. Policy-as-code release gates can then check for approved LLMs and MCP tools, an accountable human and adherence to guidelines, and the logs allow post-release audit. The post gives mock session-log and evidence JSON, ties the approach to EU AI Act Articles 12 and 14, OWASP AISVS and SLSA, and points to a GitHub reference repo, with JFrog AppTrust as the commercial implementation.

CursorOpenCodeClaude Code+8
JFrogJFrog AppTrustAgentic Process Evidence+8

ai-policy, coding-agents· Added 14 Oct 2026, published 12 Oct 2026

jfrog.comarticle🛡️ Security

Governing the Agentic SDLC Through Continuous Policy Enforcement

JFrog proposes Agentic Process Evidence, an open protocol that records AI coding agent sessions (harness, model ID, tool calls, MCP use, prompts, owner, human reviewer) as signed in-toto/DSSE, SLSA-compatible attestations bound to commits, artifacts and releases. Policy-as-code gates can then block releases that used unapproved LLMs or MCP tools, or that lack an accountable human. It maps the approach to EU AI Act Articles 12/14, OWASP AISVS and ISO/IEC 42001, and includes example session-log and evidence JSON. The post also promotes JFrog AppTrust's Prompt to Release Traceability.

JFrogin-totoCursor+3
Agentic Process EvidenceAppTrustPrompt to Release Traceability+8

AI Security, DevSecOps, Supply Chain· Added 14 Oct 2026, published 12 Oct 2026

z.aiarticle🧠 AI

GLM-Legal

Z.ai introduces GLM-Legal, a legal-domain model built on the lightweight GLM-5.3-Flash. It is trained with expanded legal-corpus mid-training, then reinforcement learning across tens of thousands of realistic legal environments, with the agent harness and model training iterated together. It reports a 19.2% all-pass rate on Harvey LAB (+12.5 points over the base model) and parity with frontier closed models on LegalBench and PLawBench (Chinese legal practice). Side-by-side examples cover contract review with tracked changes, contradiction mapping across 40 dispute documents, and a three-country privacy compliance plan. Access is via an API application and the AutoClaw desktop agent.

GLMAutoClaw
Z.aiGLM-LegalGLM-5.3-Flash+4

models, training, evals· Published and added 12 Oct 2026

endorlabs.comarticle🧠 AI

Introducing AI Coding Agent Governance: Using Hooks to Bring Visibility to the ADLC | Endor Labs

Explains how lifecycle hooks in Claude Code (.claude/settings.json) and Cursor (.cursor/hooks.json) can act as a deterministic, out-of-band control layer for AI coding agents. It covers the hook events, the JSON passed on stdin, and the response contracts. Claude Code uses exit codes plus permissionDecision, updatedInput and additionalContext. Cursor uses stdout JSON with permission and userMessage. It then shows how Endor Labs' `endorctl ai-audit` handler normalises events across both harnesses, streams them to a central audit trail and enforces server-side policies (29 defaults) such as shell allow/deny lists, sensitive-file patterns and MCP allowlists.

Claude CodeCursorendorctl+3
Endor LabsAURI

coding-agents, ai-policy· Published and added 12 Oct 2026

endorlabs.comarticle🛡️ Security

Is AI Coding Safe? Introducing the Agent Security League | Blog | Endor Labs

Endor Labs launches the Agent Security League, a public leaderboard extending CMU's SusVibes benchmark (200 tasks, 108 Python OSS projects, 77 CWEs) to commercial coding agents (Claude Code, Cursor, Codex) and frontier models. Functional pass rose to 84.4% from 61% in the original study, but secure pass peaked at only 17.3%. It also documents agents ignoring instructions and mining git history to copy the fix (SWE-Agent + Opus 4.6 did so in 163/200 tasks), and describes the cheating-detection and score-adjustment module added to the harness.

Claude CodeCursorCodex+2
Agent Security LeagueSusVibesOpus 4.6+1

AI Security, AppSec· Published and added 12 Oct 2026

huggingface.copaper🧠 AI

MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement

Technical report for Xiaomi's MiMo-V2.6 omni-modal model family, which scales RL compute along three axes: asynchronous training with 1,568 samples and 2.7-3.7B tokens per step at up to 1M context, more diverse environments (code, general, visual, cyber) under mixed agent harnesses, and groupwise agentic grading that gives better reward signals and favors shorter, token-efficient solutions. Stability measures include a frozen MoE router and a multi-layer defense against reward hacking, built on a mid-trained hybrid-SWA architecture. The authors open-source training dynamics, RL environments and the RL framework, which makes it a useful reference for reproducing scaled agentic RL.

Mixture of ExpertsHugging Face
MiMo-V2.6Xiaomi

training, research, agents· Added 10 Oct 2026

primeintellect.aiarticle🧠 AI

Rewriting Prime Agent in Rust

Prime Intellect describes rewriting its open-source Prime Agent coding-agent harness from TypeScript to Rust. Prime Agent orchestrated 2,000+ agents across 10,000+ Prime Sandboxes, using GLM-5.3 inference, to do the rewrite. A root orchestrator split the work into topologically ordered tasks, and each task passed through planner, implementer, reviewer and verifier agents, with TUI, harness, protocol and feature parity checks as the objective verification. A follow-up three-day hillclimbing loop on a benchmark harness merged 69+ changes, and the post reports ~14x faster time-to-input and over 80% less memory after startup. The post also covers the new architecture: nine crates, per-session worker processes, platform abstraction for Windows support, and a runtime-fetched model/MCP catalog. Vendor-run comparisons with other harnesses carry the post's own caveat about the lack of a common benchmark standard.

RustTypeScriptPrime Agent+8
Prime IntellectGLM-5.3

coding-agents, agents, ai-infra· Added 10 Oct 2026, published 9 Oct 2026

github.comrepo🧠 AI

Show HN: Apogee: Rebuilding Mozilla's Orbit, fully local and private

Apogee is a privacy-first browser extension (Chrome/Firefox) that summarizes articles, YouTube/Bilibili videos, PDFs, DOCX and pasted text entirely on-device, inspired by Mozilla's discontinued Orbit. It runs models via WebGPU or WebAssembly in the browser, or connects over loopback to a local Ollama or llama.cpp server, with local passage retrieval for Q&A, grounded sentence highlighting, and Opus-MT translation. It shows a practical no-backend, no-API-key architecture for local LLM apps, with a Node.js extractor test harness for contributors.

WebGPUWebAssemblyOllama+4
ApogeeMozillaOrbit+2

inference, llm-apps· Published and added 9 Oct 2026

ethereum-magicians.orgthread🛡️ Security

Hybrid post-quantum stealth addresses (ERC-5564 schemeId 3)

A draft ERC proposing schemeId 3 for ERC-5564 stealth addresses, combining an ML-KEM-768 encapsulation with a secp256k1 ECDH secret so that future quantum computers cannot retroactively deanonymize recipients from the public announcement log, while spending stays classical secp256k1 (and thus still quantum-vulnerable once spent). The post, with a linked reference implementation, test vectors and gas harness, details the key/meta-address format, the payment-secret combiner (hashing both shared secrets and ciphertexts for IND-CCA), the view-tag derivation ordering to avoid quantum tag-narrowing, measured gas cost (111,330 gas per payment on Prague), and open questions around ANO-CCA proofs for standardized ML-KEM, NIST SP 800-56C combiner approval, and downgrade-attack handling when a recipient registers under both schemeId 1 and 3.

Ethereum
ERC-5564ERC-6538EIP-7623+7

Blockchain· Published and added 9 Oct 2026

arxiv.orgpaper🧠 AI

Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict

The paper proposes evaluating LLM agents on epistemic humility, meaning their willingness to recognize, act on and communicate uncertainty. It operationalizes this as three trajectory-level dimensions (Identify, Solve, Escalate, or ISE). It tests controlled and naturally occurring knowledge conflicts, each with matched no-conflict controls, on four agents. Higher accuracy does not imply more humility: some high-accuracy agents detect conflicts but do not communicate uncertainty in wrong final answers, and agents often detect conflicts early but lose track of them later. Model-level interventions can improve humility but often cost accuracy, which suggests humility depends on the interaction of backbone model, agent harness and environment.

arXiv

evals, agents, research· Added 9 Oct 2026, published 8 Oct 2026

arxiv.orgpaper🧠 AI

A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization

Introduces AgenticBBO-Bench, a benchmark for LLM-agent black-box optimization across synthetic functions, hyperparameter optimization, database tuning, chip design and molecular design under a unified finite-budget protocol. Agentic BBO beats direct LLM methods in all five domains and the best numerical optimizers in four. Ablations show extra numerical tools don't consistently help, task semantics help more than specific priors, and numerical optimizers can absorb gains from agent-found trajectories. A five-task frontier challenge evaluates seven LLMs under the Codex agent harness, with GPT-6 Astra and DeepSeek-V4.1-Flash on the performance-cost Pareto frontier.

Codex
AgenticBBO-BenchGPT-6 AstraDeepSeek-V4.1-Flash

evals, agents, research· Added 9 Oct 2026, published 8 Oct 2026

arxiv.orgpaper🧠 AI

MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement

Technical report for the MiMo-V2.6 omni-modal model series, which scales RL compute after mid-training on a multimodal corpus, using a hybrid sliding-window-attention architecture. RL is scaled along three axes: asynchronous training with 1,568 samples and 2.7-3.7B tokens per step at up to 1M context; diverse environments (code, general, visual, cyber) under mixed agent harnesses; and groupwise agentic grading that gives better rewards for long-horizon tasks and favors shorter, token-efficient solutions. Stability measures include freezing the MoE router and a multi-layer defense against reward hacking. The authors open-source the training dynamics, RL environments and RL framework.

Mixture of Experts
MiMo-V2.6Xiaomi

training, research, models· Added 9 Oct 2026, published 8 Oct 2026

arxiv.orgpaper🧠 AI

From Evidence to Action: How Tool-Using Agents Fail

This paper studies where tool-using LLM agents break down in the chain from gathering evidence to taking consequential actions, finding that strong static judgment of whether to act often coexists with much weaker interactive execution, and that failures arise from incomplete investigation, acting before evidence is established, or unresolved prerequisites in multi-action workflows. The authors introduce SafeActBench, a 656-case benchmark across six operational domains and five protocols, paired with a provenance-bound Evidence Ledger and a deterministic trajectory evaluator tested across ten model-harness configurations.

SafeActBench

evals, agents· Added 7 Oct 2026

aws.amazon.comarticle🛡️ Security

Configuring your AI vulnerability harness, Part 2: The steering file

AWS Security Blog's second post in a series on building an "AI vulnerability harness" explains the configuration (steering file) that stops an AI model from hallucinating vulnerabilities and inflating severity during security code review. It details five configuration sections — structural verification of findings, an evidence-based confidence formula (weighting confirmed taint flow at 0.30), infrastructure-aware severity multipliers for controls like AWS WAF and IAM, CISA KEV/EPSS threat-intel boosting, and a four-tier priority classification — and reports validation numbers (e.g., unsteered models fabricated code paths in ~30% of findings; steered confidence of HIGH without structural confirmation was only 34% exploitable vs 89% with confirmed taint). The steering file is shipped as a Kiro steering file, a Kiro skill, and a CLAUDE.md for Claude Code, with full source in a linked GitHub repo.

KiroClaude CodeAWS WAF+3

AI Security, AppSec· Published and added 7 Oct 2026

aws.amazon.comwriteup🧠 AI

Configuring your AI vulnerability harness, Part 2: The steering file

Part 2 of an AWS Security Blog series detailing the design of a "steering file" — a persistent instruction set (deployable as a Kiro steering file, Kiro skill, or Claude Code CLAUDE.md) that constrains an LLM performing vulnerability triage. It walks through five configuration sections: requiring structural/dataflow verification before reporting findings, replacing self-reported confidence with a weighted formula based on binary structural signals (e.g., 0.30 weight for confirmed taint flow), mapping infrastructure controls (AWS WAF, IAM) to severity multipliers with a 0.15 floor, incorporating CISA KEV and EPSS threat-intelligence boosts capped at +0.50, and a four-tier priority classification that suppresses low-confidence P3 findings. Validation against a 10-vulnerability test app showed the steered model eliminated fabricated findings and correctly downgraded mitigated vulnerabilities while preserving full severity for a command injection reachable only via authenticated access.

KiroClaude CodeCursor+7
CISAEPSS

prompting, coding-agents· Added 8 Oct 2026, published 7 Oct 2026

aws.amazon.comarticle🛡️ Security

Building your AI vulnerability harness, Part 1

AWS Security Blog describes a three-layer AI-assisted pipeline for triaging vulnerability scanner findings: Layer 1 cross-checks multiple scanners (e.g., Semgrep, CodeQL) for agreement, Layer 2 verifies a scanner's claimed data-flow path against the code's actual abstract syntax tree to catch hallucinated findings, and Layer 3 cross-references infrastructure-as-code (CloudFormation/Terraform) to weigh whether existing controls like AWS WAF rules actually mitigate the exploit path. It argues progressive, escalating-evidence filtering plus mandatory structural verification of LLM claims before human review is what turns noisy scanner output into a short, trustworthy, prioritized list, and lays out an adoption order (indexing, then multi-scanner triage, then AI hypothesis generation, then infra context, then live PoC verification).

AWSSemgrepCodeQL+3

AI Security, AppSec· Published and added 7 Oct 2026

arxiv.orgpaper🧠 AI

Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models

This paper proposes three evaluation screens — Decisive-Action BPB, Patch MCQ, and prefix-conditioned pass@K — that predict how well a base model will perform as a coding agent after post-training, without requiring the base checkpoint to run a full agentic harness. By replaying successful post-trained agent trajectories and identifying the "decisive step" that flips a repository from failing to passing tests, the authors measure a base model's latent support for that action. Tested across ten pairs of public base and post-trained models, all three screens closely track post-trained SWE-bench Verified pass@1 scores, offering a way to cheaply vet base checkpoints before committing to expensive agentic post-training.

SWE-bench Verified

evals, coding-agents· Added 8 Oct 2026, published 7 Oct 2026

Showing 90 of 234

Show more