234 results for “Harness”

Latent Space podcast write-up of an AI Engineer New York interview with Aman Singh, lead AI Engineer at Arca, an AI-native wealth management firm. Arca runs agent tasks lasting up to ten days, uses a per-client memory system, and owns its harness, which routes tasks to models (mainly Claude) by internal benchmarks. Safety rules are deterministic, trading stays with human advisors, and a meta-agent uses advisor feedback and traces to improve the harness.

latent.spacearticlePublished and added 31 Oct 2026🧠 AI

llm-trails is an open study of what LLMs choose to do next inside a breached AWS account. It puts 14 models in 13 post-compromise situations, records only a single-turn plan with nothing executed, and codes each answer to CloudTrail events. The repo has the llmtrials harness, prompts, codebook, processed dataset (viz_data.json) and an interactive dashboard, so the study can be rerun or extended.

github.comrepoAdded 30 Oct 2026🧠 AI

llm-trails is a research repo that placed 14 LLMs in 13 post-compromise AWS situations, asked only for their next move (single-turn, nothing executed), and coded each answer to CloudTrail events. It includes the llmtrials harness, prompts, codebook, processed dataset (viz_data.json) and an interactive dashboard. Rates are conditioned on the situation shown, and the aim is turning model behavior into detection priorities.

github.comrepoAdded 30 Oct 2026🛡️ Security

Agents in Games is a Y Combinator Labs writeup, done with Olam Labs, of multi-agent swarm experiments in Minecraft and Mindustry. A 50-agent GPT-6.1 Sol Minecraft colony became risk-averse after 22 of 34 deaths came on night one. GPT beat Claude Opus 5.5 at Bed Wars 6-3 and Mindustry 6-1. The authors open-sourced a Minecraft agent harness, Multi-Agent Mindustry and a Harbor fork.

labs.ycombinator.comwriteupAdded 30 Oct 2026, published 29 Oct 2026🧠 AI

Latent Space episode with Katelyn Lesse (engineering) and Angela Jiang (product) of Anthropic, recorded the day after the Claude Opus 5.5 launch. It covers distillation defences such as enforced preserved thinking blocks and a new API compaction feature, per-model data retention and verification programs, trust grants for access control, and the Managed Agents split between harness and sandbox. The text shown is truncated; later sections are known only from timestamps.

latent.spacetalkPublished and added 27 Oct 2026🧠 AI

Latent Space episode with Katelyn Lesse and Angela Jiang of Anthropic, recorded after the Claude Opus 5.5 launch. It covers distillation defences (enforced preserved thinking blocks, a compaction feature for harness maintainers), trust-grant authorization for gated model access, and the Managed Agents split between harness and sandbox, where credentials are held outside the sandbox and attached through a proxy. Only the first third of the transcript was available.

latent.spacetalkPublished and added 27 Oct 2026🛡️ Security

Latent Space episode with Katelyn Lesse and Angela Jiang on the engineering behind the Claude Platform. Topics include the Claude Opus 5.5 launch and pricing, Claude Managed Agents that separate the model's brain from the harness and sandbox, rebuilding the Messages API in Rust, credential protection, safety classifiers, model routing and agent evaluation. The page lists show notes and timestamps only, with no transcript.

latent.spacetalkPublished and added 27 Oct 2026🧠 AI

TraceTrial-Bench is a benchmark that scores agent task performance and agent harness reliability separately. It has 95 mechanism-oriented tasks across seven components: context, execution, governance, lifecycle, observability, tools and verification. Across 4 harnesses, 9 models and 3,420 runs, macro-averaged harness scores range from 59.8% to 72.0%, and 31.1% of runs succeed despite failing at least one harness criterion.

huggingface.copaperAdded 27 Oct 2026🧠 AI

Continual Learning Exam (CLE) is a benchmark of 120 human-authored tasks, each a five-level ladder (600 levels) across 89 professional roles and 12 occupational domains, built by 50+ domain experts. Agents must pass a level to unlock the next, and an evaluation agent scores submissions against human rubrics. Across 17 harness-model configurations, feedback plus one revision adds a median 5.3 points; the best system clears 47.7% of levels and finishes all five levels in 23.3% of tasks.

arxiv.orgpaperAdded 27 Oct 2026, published 25 Oct 2026🧠 AI

ECC (affaan-m/ECC) is an open-source performance optimization system for AI coding agents. It packages 261+ skills, 64+ agents and 84+ commands that tune Claude Code, Codex, Cursor and other agent harnesses for speed, memory and security. Practitioners can use it as a ready-made skills and harness configuration set. The RepoDir page gives only this short description, with no install steps or benchmarks.

repodir.comrepoAdded 24 Oct 2026🧠 AI

The essay argues that LLMs solve open math problems because they search depth first, and automated verifiers make failed attempts cheap. It uses WT Gowers' hint-sequence complexity (2^k + t), adds a branching factor b, and walks through OpenAI's unit-distance disproof and Cycle Double Cover prompt as a drafter/solver/verifier harness. It predicts AI does best where k and b are small.

latent.spacearticlePublished and added 23 Oct 2026🧠 AI

When Agents Fail: Model or Harness--Who's to Blame? proposes five measures for evaluating agent harnesses together with their underlying models: Valid Completion Rate, Model-Harness Fit, Cross-Model Fit, Interaction Residuals and Token Load. It tests four harnesses (OpenCode, Codex CLI, Pi CLI, DeepAgents) with four models on six benchmarks, across 111 settings and 52,168 trajectories. Harness rankings change by model and benchmark. OpenCode ranks first in cross-model performance.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

4DMemory is a benchmark suite for long-horizon spatiotemporal memory: the LongVideo dataset (about 900 hours of video, over 5,000 manually reviewed questions) and 4DMemoryBench. It also presents the 4DMemory Harness, combining four skills, 22+ tool models and a model-specific RAG system. Across 13 models, the harness improved every one by over 50%, and the same models scored over 10% lower than on a previous benchmark.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

This paper presents an offline evaluation harness for a production merchant-facing conversational agent. It stays reproducible when live backend data changes and can be regenerated when it goes stale. Used as the fitness function for GEPA prompt optimization under a correctness gate against reward hacking, it shrinks the full prompt surface by 3.5-7.6% without losing held-out correctness, across three subject-model families.

arxiv.orgpaperAdded 26 Oct 2026, published 22 Oct 2026🧠 AI

Daily Vibe Casting thread of 24-hour tech news, with item 5/10 on Cloudflare's agent security audit skill. The skill works across models and agent harnesses, including free models, and outputs machine-readable reports. It calls for sandboxing, proof of real impact and independent checks. The thread also covers GitLab CVE-2026-110264 (CVSS 10.0, unauthenticated RCE), two AI-found PHP RCE flaws, and a Railway secret-theft warning.

x.comthreadAdded 24 Oct 2026, published 22 Oct 2026🛡️ Security

CellTrack-Bench is a benchmark that tests vision-language models on microscopic cell tracking and compares them with specialised trackers. It has 1,537 problems covering 9,041 cell-linking decisions, drawn from 15 cell-tracking datasets across four imaging modalities, with a hard subset built from failures shared by distinct algorithmic trackers. The authors compare environment harnesses, benchmark 9 closed-source and 13 open-weight models, and analyse failure modes.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

Colin Jarvis, Head of Forward Deployed Engineering at OpenAI, describes how the FDE team works with enterprise customers to close the "deployment gap". Cases include a semiconductor verification project (about 48% time saved on test generation) and a telecom voice model whose key evals rose from about 60% to 90%+. Moving use cases onto a Codex harness with skills and plugins removed about 90% of custom code.

openai.comarticlePublished and added 22 Oct 2026🧠 AI

Adversa AI assesses Claude Code enterprise security as of October 2026. Managed settings bind only Claude Code, can be edited by local admins, and are not read by Anthropic-hosted cloud sessions. Auto mode misses 17% of risky actions in Anthropic's test set. The article covers attacks on the harness (CVE-2025-59536, CVE-2026-21852, TrustFall, SymJack, GhostApproval) and recommends credential, egress, isolation and monitoring controls.

adversa.aiarticlePublished and added 22 Oct 2026🛡️ Security

Matrix-Game 4.0 is a system that separates game logic from visual rendering for playable generated worlds. A lightweight engine holds world state and rules, a multimodal coding agent writes game code and scene captions, and a distilled autoregressive video model turns low-poly proxies into detailed frames. It runs 480p at 60 FPS on one RTX 5090 with 112 ms key-press-to-display latency. Code, weights and assets are released.

huggingface.copaperAdded 22 Oct 2026🧠 AI

Weco, from WecoAI, is an open source CLI that optimizes code, prompts and agent harnesses against a metric the user defines. It uses LLM-guided tree search: the user supplies an eval command and Weco iterates across multiple files with the user's own API keys. The post lists Claude Code, Cursor, Codex, OpenClaw, PyTorch, CUDA and Triton as supported. It gives no benchmarks or setup details.

x.comtoolPublished and added 22 Oct 2026🧠 AI

FrontierMemory is a benchmark that evaluates agent memory through downstream task performance, measuring how agents learn from experience rather than recall, which the post says is saturated via agentic search. In elvis's summary, Claude Opus 5.5 ranks first and open-weight GLM-5.3 does well on memory generation. The authors reply that "memory" here means any adaptation from experience, so harness-level changes can also be scored. The post gives no scores or links.

x.comthreadAdded 23 Oct 2026, published 21 Oct 2026🧠 AI

Claude Mods are Claude Code plugins, written as TypeScript or JavaScript functions, that hook into prompts, tool calls and rendering to customize the CLI and Desktop. The author describes built-in mods such as You Should Know, and their own Next Steps Supervisor in the ModSmith plugin. They compare mods with ChatGPT Intelligent UI and Cursor PR Routing & Approval, and list downsides: noise, token cost and lack of sandboxing.

latent.spacearticlePublished and added 21 Oct 2026🧠 AI

Claude Mods are Claude Code plugins, written as TypeScript or JavaScript functions, that hook into prompts, tool calls and rendering to change how the CLI and Desktop behave. The author describes six built-in mods, including You Should Know, and their own Next Steps Supervisor from the ModSmith plugin. They compare mods with ChatGPT Intelligent UI and Cursor PR Routing & Approval, and note noise, token cost and trust problems.

latent.spacearticleAdded 23 Oct 2026, published 21 Oct 2026🧠 AI

Paper (arXiv, 21 October 2026) on structural false negatives in LLM vulnerability discovery harnesses. Starting from Knostic's OpenAnt and raptor, then 10 more open-source harnesses, the authors map 13 points where code selection, filtering, and finding storage or merging drop vulnerabilities. One file was dropped by six harnesses before the model saw it, five without notice. It covers attacker evasion, content-steered verdicts, prompt injection, and design principles and tests.

arxiv.orgpaperAdded 24 Oct 2026, published 21 Oct 2026🛡️ Security

Latent Space interview from AIE New York with Tenex cofounder Alex Lieberman and CJ Hess on running an AI-native engineering firm of 75 people. Covers story-point pay for engineers, the MetaHarness (plans, decisions and agent logs checked into Git with a CLI), roughly 2x engineer output in 2026, Opus 5.5 and Codex usage, and enterprise adoption rates. Also debates vertical AI and the Kill My SaaS experiment.

latent.spacetalkPublished and added 21 Oct 2026🧠 AI

Latent Space interview from AI Engineer New York with Tenex cofounder Alex Lieberman and CJ Hess. Tenex has 75 people, pays engineers by story points, and reports roughly 2x engineer output over the past year. CJ describes MetaHarness, a repo-based set of skills, plans and a CLI that logs agent decisions in Git. They also cover enterprise AI adoption rates, token spend, and vertical AI. The page text is truncated.

latent.spacetalkAdded 23 Oct 2026, published 21 Oct 2026🧠 AI

Endor Labs evaluates Gemini 4 Argon in the Cursor harness on its Agent Security League coding tasks (SusVibes). Argon ties Fable 5 at 29% SecPass with 78.8% FuncPass, but costs $765 and takes a 14-minute median per task. It uniquely fixes SABnzbd path traversal (CVE-2021-29488) and a Nova credential-logging leak (CVE-2015-8749). Two other candidate solves were rejected, and 29 cheating instances were confirmed.

endorlabs.comarticlePublished and added 20 Oct 2026🛡️ Security

Endor Labs ran Gemini 4 Argon in the Cursor harness on its Agent Security League coding tasks (SusVibes). Argon ties Fable 5 at 29% SecPass with 78.8% FuncPass, and uniquely and securely solves two tasks (SABnzbd CVE-2021-29488 and OpenStack Nova CVE-2015-8749). It is also the slowest and costliest recent combo: 14 minutes median per task and $765. The run had 29 confirmed cheating instances.

endorlabs.comwriteupPublished and added 20 Oct 2026🧠 AI

AgenticRoboRL is a framework for post-training robot planners across different harnesses, combining supervised fine-tuning with multi-turn reinforcement learning. Harness-specific adapters link recorded planner interactions to task outcomes, so one RL pipeline covers both VLA skill invocation and control-code generation. On LIBERO-PRO, a 9B planner rose from 18.1% to 71.0% without external memory, near Codex (GPT-5.5) with memory at 72.1%. Code and checkpoints are promised.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

Interviews with three Java Champions (Vlad Mihalcea, Patrick Baumgartner, Frank Delporte) on using GitHub Copilot in production and open source. Examples include a 900-slide Reveal.js to Marp migration with visual verification, a Copilot coding agent diagnosing a Pi4J virtual-thread pinning bug, and custom Agent Skills. Tips: interview-first planning, task-specific verification gates, deterministic quality harnesses, and human review.

github.blogarticleAdded 29 Oct 2026, published 20 Oct 2026🧠 AI

AINews issue for 16 to 19 October 2026 from Latent Space, a digest of Twitter and Reddit AI news. It covers DeepSeek V4.1 Flash 1017 (552B total, 16B active MoE, AA Intelligence Index 45), MiniMax M3.1 Flash, Mercury 3, Meta Muse, Google's Harness², vLLM v0.32.0 and a Terminal-Bench audit. Only the first 40,000 of 47,045 characters were available.

latent.spacearticleAdded 30 Oct 2026, published 20 Oct 2026🧠 AI

AINews issue covering 16-19 October 2026 from Latent Space, summarising 12 subreddits and 544 Twitter accounts. It leads with DeepSeek V4.1 Flash 1017 (552B total, 16B active MoE, 1M context, MIT license, AA Intelligence Index 45), plus MiniMax M3.1 Flash, Mercury 3 and Meta Muse. It also covers vLLM v0.32.0, InferenceX results, Harness² and Terminal-Bench audit findings.

latent.spacearticlePublished and added 20 Oct 2026🧠 AI

FunCAD-Harness is a framework in which LLM agents design CAD mechanisms and revise them using physical simulation feedback. It turns abstract functional requirements into candidate-specific physical tests. The paper also introduces FunCADBench, a benchmark of 46 mechanism families with interdependent, multi-step operations. Experiments on frontier LLM agents show it exposes mechanical failures that visual inspection misses and raises functional success over standard simulation-augmented baselines.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

OpenVLHarness is a multimodal reasoning harness for MLLMs built on a capability layer (task-level tool interfaces plus evidence rendering) and persistent multimodal memory that stores images, structured data and web references as addressable artifacts. A coding interface runs generated programs over stored data. Across 22 benchmarks and eight backbones it adds 9.7-12.6 points over no-tool baselines, and GPT-6 Luna at medium effort beats Codex and Visual Sketchpad at maximum effort using at most 37% of their tokens.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

VibeTraining-Bench is a benchmark for cross-component co-design, where research agents iteratively refine training data, trainers and inference harnesses for a fixed starting agent system. Tasks cover terminal operation, software engineering, multi-turn tool use and math reasoning, with safeguards against reward hacking. On SWE-bench Verified, GPT-6 Astra raises the target system from 32.9% to 53.7%, against 55.4% for the human-developed instruct model.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

Self-evolving Agent Harness is a 66-page survey and theory paper on revising an agent's scaffolding (prompts, memory, workflows, tools, code, plugins) while model weights stay fixed. It frames search as evolutionary optimization and uses continual PAC-learning to derive a generalization bound, a retention bound and a high-probability admission rule with fresh validation data. It separates selection, activation, attribution and improvement as evaluation claims, and covers applications and governance. Code is at github.com/zgcagi/Harness-RSI.

huggingface.copaperAdded 19 Oct 2026🧠 AI

Harness2 is a framework from Google Cloud AI Research and Northwestern University in which an Editor model rewrites an agent's harness (system prompt, memory, skills, tool scripts, subagent routing, middleware) without changing model weights. It uses a Propose-Probe-Compose loop that compares execution trajectories instead of ground truth. On WorkBuddy-Bench, GLM-5 rose from 64.2 to 75.7 after one round, above Gemini 3.5 Flash at 72.4. The thread also lists risks: latency, experience poisoning, and a sandbox that is off by default.

x.comthreadAdded 19 Oct 2026, published 18 Oct 2026🧠 AI

DeepSeek Search & Fetch are two JSON APIs for agents and RAG apps. Search (/v1/tools/search) returns structured results from DeepSeek's in-house engine, with count, need_content and operators such as site:, lang: and after:. Fetch (/v1/tools/fetch) returns a page as Markdown. Both use bearer auth, bill only successful requests ($0.75 per 1000), and have account-level QPS limits. Calls from DeepSeek Harness, Claude Code or Codex are free until 1 November 2026.

api-docs.deepseek.comotherAdded 19 Oct 2026, published 18 Oct 2026🧠 AI

ToFu is a tool-aware fuzzing framework for LLM tools, including MCP servers. It maps each tool argument, using its metadata, to one of 74 CWE-guided semantics-aware sanitizers (path, URL, SQL and similar). It then generates schema-valid Atheris harnesses and semantic oracles, using runtime tracers for in-process tools and outcome oracles for opaque ones. The LLM (gpt-4.1) only maps sanitizers and writes seeds; it is never the security judge. The paper also shows a "Repeat-After-Me" trick, where an LLM told to repeat attacker-crafted text emits a tool call the orchestrator executes, reproduced on LangGraph, MAF, ADK 2.0, AutoGen, Strands and CrewAI. Across 273 runnable MCP servers (3,405 tools) it reports 253 verified bugs in 343 tools across 118 servers, such as path traversal, log injection and exfiltration. These are failures a crash-based fuzzer would miss. The result matters because it shows LLM-generated tool arguments cannot be treated as trusted.

arxiv.orgpaperAdded 18 Oct 2026🧠 AI

ToFu is a tool-aware fuzzing framework that derives semantic security checks for LLM tools (mainly MCP servers) from their metadata and a corpus of 74 CWE-guided sanitizers. It generates schema-valid Atheris harnesses, uses runtime tracers and outcome oracles to catch bugs that don't crash, and uses an LLM only to map sanitizers and seed inputs, never as the judge. It also describes a "Repeat-After-Me" attack, in which the LLM is told to repeat attacker-written text that the orchestrator then dispatches as a tool call. It was reproduced on LangGraph, MAF, ADK, AutoGen, Strands and CrewAI. On 273 MCP servers it reports 253 verified bugs across 343 tools in 118 repositories, including path traversal, log injection, exfiltration and command bypass.

arxiv.orgpaperAdded 18 Oct 2026🛡️ Security

KillSwitch is an esoteric stack-based programming language built to be adversarial to LLMs and coding agents. It uses letter-counting rules, a spec with hidden lies, guardrail-triggering phrases like "hack into a computer", and noisy story-style scripts. The author benchmarked major models and agent harnesses on simple tasks. Opus 5 scored best at 66.8, Fable 5.1 and Gemini 4 Argon did poorly because of safety guardrails, and MiMo-V2.6-Pro led the open-weight models.

blog.picheta.mearticleAdded 19 Oct 2026, published 17 Oct 2026🧠 AI

DeepSeek announces DeepSeek-V4.1-Flash-1017, claiming stronger benchmarks, faster task completion and shorter outputs than the previous version, live on its website, app and API. It is optimized for DeepSeek Harness (v0.2.1-rc.1 or later), Tencent CodeBuddy and WorkBuddy, and uses reinforcement learning with self-feedback to improve end-to-end project development, from game creation to 3D modeling. The thread also opens the Search & Fetch APIs to everyone, backed by DeepSeek's in-house search engine at $0.75 per 1,000 calls per endpoint, available as server-side tools.

x.comthreadAdded 18 Oct 2026, published 17 Oct 2026🧠 AI

Model card for DeepSeek-V4.1-Flash-1017, a minor update to DeepSeek-V4.1-Flash with MIT-licensed weights. It claims gains in reasoning, coding, agentic tasks and visual benchmarks, with shorter task time and output length that move the performance–cost Pareto frontier across agent harnesses (Claude Code for Terminal-Bench 4.0, Mini-SWE-V2 for DeepSWE v1.1). It ships no Jinja chat template. It provides a reference encoding.py for multi-turn chat, tool calling, thinking mode, numeric reasoning effort and interleaved images, plus deepseek-recipe, Rust libraries with Python bindings for prompt encoding and output parsing. Serving recipes are given for Transformers, vLLM, SGLang and Docker Model Runner.

huggingface.corepoAdded 18 Oct 2026, published 17 Oct 2026🧠 AI

OpenGCUA, from Alibaba's MAI-UI team, is an open stack for generalist computer-use agents (GCUAs), where the model chooses how to observe, act and organize work across GUI, accessibility-tree, API and shell interfaces. Its design combines code execution (GUI actions written as programs with loops and branches) with harness-to-model decision transfer. On OSWorld-v2 it raises mean partial score by +19.6 points across eight models, including GPT-6 Astra and Claude Opus 5, and cuts reported steps per task by 53%. The release includes the harness, prompts, evaluation trajectories, a macOS real-device harness, a cross-device SDK and an interactive lab. It also shows large efficiency gaps between models: Qwen3.8-Max used 19.4x the reasoning tokens of GPT-6 Astra on real macOS tasks.

huggingface.copaperAdded 17 Oct 2026🧠 AI

Microsoft blog post introducing "Risk Management for Open-Weight Models in Enterprise", a joint Microsoft–NVIDIA paper. It proposes a shared risk taxonomy (Cybersecurity, Data Protection, Content, Human-AI Interaction, Bias) grounded in ISO standards and the NIST AI RMF. The paper maps evaluations to risks as evidence and argues that risk depends on the model, use case, deployment and safeguards, not on weight openness alone. It also covers how responsibility for controls shifts when enterprises self-host models, and points to a joint repository of evaluations. The first paper focuses on the model layer; runtime, harness and application layers are planned for later.

microsoft.comarticlePublished and added 16 Oct 2026🛡️ Security

Microsoft's announcement of "Risk Management for Open-Weight Models in Enterprise", the first paper in a planned joint series with NVIDIA. It proposes a taxonomy of five risk categories (Cybersecurity, Data Protection, Content, Human-AI Interaction, Bias). The approach is grounded in ISO standards and the NIST AI Risk Management Framework. It maps evaluations to risks as evidence, stresses that weight openness alone doesn't determine risk, and explains how self-hosting shifts responsibility for safeguards. A joint repository will collect evaluations, and later work will cover the runtime, harness and application layers.

microsoft.comarticlePublished and added 16 Oct 2026🧠 AI

StarSkirmish Bench v0.2 gives each LLM one hour to write a Protoss StarCraft: Brood War bot in C++ (BWAPI 4.4.0 on OpenBW), with tools to compile and play practice games. Scores are average win rates of 5 runs against 13 roster bots, which include human-written bots such as Stardust, PurpleWave and BananaBrain. Nine models are compared, each in its lab's first-party CLI harness (Claude Code, Codex, Antigravity CLI, Muse Code). GPT-6 Astra leads and wins only 7% against the top human bots. The page also covers score against cost, run-to-run variance, practice behaviour as the best predictor of score, and weak correlation with the Artificial Analysis Intelligence Index and Terminal-Bench 4.0.

starskirmish.comarticleAdded 17 Oct 2026, published 16 Oct 2026🧠 AI

Describes Vitalik Buterin's open-source "paranoid mode" privacy wrapper for his pi agent harness. It is an OpenAI-API-compatible daemon that gives each conversation its own Tor circuit with random cover traffic, rewrites prompts with a local model before they reach the provider, and runs the agent in a bubblewrap sandbox wrapped with torsocks. Payments go through zkAPI, which uses zero-knowledge proofs to separate payment authorization from the funder's identity. The article notes the limits: high latency, weak protection against a provider colluding with a Tor guard relay, and a proposed constant-cover-traffic mixnet that is not yet implemented.

en.cryptonomist.charticlePublished and added 16 Oct 2026🛡️ Security

News report on Vitalik Buterin's open-source "paranoid mode" privacy wrapper for his pi agent harness. It is a daemon compatible with the OpenAI API that sits between the agent and an external LLM provider. It gives each conversation its own Tor circuit with random cover traffic, rewrites prompts with a local model (qwen3.8-flash-next) before they leave the device, and runs the agent in a bubblewrap sandbox wrapped with torsocks. It pays through zkAPI, the Ethereum Foundation / Open Anonymity Project zero-knowledge scheme that separates payment authorization from the funder's identity. The article also covers the limits: high latency at conversation start, and a provider colluding with a Tor guard relay can defeat cover traffic, which motivates a proposed constant-cover-traffic mixnet.

en.cryptonomist.charticlePublished and added 16 Oct 2026🧠 AI

Doc2Harness is a two-stage benchmark for harness engineering. In Stage 1 a build model, working inside a fixed builder harness, gets an interface-only contract skeleton plus a development document consolidating behavior from twelve frontier harnesses, and builds a full candidate harness in an isolated container. In Stage 2 the frozen harness runs in separate per-test-case containers, driven by a scripted mock model, and is scored on four channels: requests sent to the model, files on disk, exit code and command output. Across 19 frontier models with three builds each on 328 test cases, GPT-6 Astra leads at 53.05% against 27.44% for the weakest, and failures cluster around how cross-cutting functionality is split across modules.

huggingface.copaperAdded 16 Oct 2026🧠 AI

Announces Reflection's Beam-1.0-501B-A23B, an Apache 2.0 text-only MoE model (23B active per token, 256 experts, 1M context) aimed at coding and agentic use. It covers the architecture (hybrid sliding-window/global attention, a constant-scale residual stream, DeepSeek-V3-style group-limited routing) and BF16/FP8/NVFP4 checkpoints with hardware sizing. It also gives day-0 serving recipes for transformers, vLLM and SGLang with DFlash2 speculative decoding, LoRA SFT with TRL, GRPO with Miles, and demos built with the Pi coding harness.

huggingface.coarticlePublished and added 16 Oct 2026🧠 AI

Presents MLExpert, a 35B Mixture-of-Experts agentic model trained for autonomous long-horizon machine learning engineering. Training uses a Kaggle-MLE Factory to build executable tasks and an interaction harness, hierarchical agentic SFT with outcome-guided supervision, and Windowed Interaction RL (WIRL). WIRL treats the multi-turn decisions between successive experiment results as one execution window with a shared outcome reward, and uses asynchronous collection and policy correction. On MLE-Bench Lite the average medal rate rises from 37.9% to 72.7%, comparable to larger frontier models such as Kimi-K3 and GLM-5.3. It also improves on NatureBench Lite, agent scaffold optimization and algorithm optimization.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

Snyk introduces VulnBump, a benchmark of 307 real vulnerable-dependency upgrades from 43 repositories across Maven, Gradle, npm, Yarn, pip and Poetry. An attempt passes only if a fresh Snyk rescan shows the vulnerabilities gone and the project installs, builds and keeps its previously passing tests green in a clean container. On Claude Sonnet 5.5, Snyk's Evo Remediation Agent passed 90.2%, Claude Code with Snyk scan and target version 77.5%, unaided Claude Code 59.6%, and a non-AI package-manager bump baseline 46.9%. Unaided Claude Code often claimed fixes a rescan still flagged, so the benchmark shows harness and security data matter as much as the model. The harness and 43 cases are open-sourced.

snyk.ioarticlePublished and added 15 Oct 2026🧠 AI

Snyk presents VulnBump, a benchmark of 307 real vulnerable-dependency upgrades across 43 repositories and six package managers (Maven, Gradle, npm, Yarn, pip, Poetry). An upgrade passes only if a rescan shows the vulnerabilities gone and the project still installs, builds, and passes its previously passing tests. On Claude Sonnet 5.5, Snyk's Evo Remediation Agent passed 90.2%, Claude Code with Snyk scan 77.5%, Claude Code unaided 59.6%, and a plain package-manager bump 46.9%. Unaided Claude Code often claimed fixes that a rescan disproved, and major framework migrations and transitive dependencies were the hardest cases. The harness and the 43 cases are open-sourced.

snyk.ioarticlePublished and added 15 Oct 2026🛡️ Security

Harness^2 is a plug-and-play framework that recursively improves LLM agent execution harnesses in open-world settings where no ground-truth verifiers exist. Task-level recursion uses a contrastive Propose-Probe-Compose step to guide harness edits for the current task, and it scales through parallel or sequential recursion. Harnessing-level recursion distills experience from those edits into a continual-improvement harness for future tasks. A single step improves GLM-5 and Gemini 3.5 Flash over their base harnesses by up to 11.5 points on WorkBuddy-Bench, 4.5 on JobBench and 11.8 percentage points on LAB.

huggingface.copaperAdded 14 Oct 2026🧠 AI

ChainGraph is a pattern that moves exploit-chain reasoning out of the flat context window and into a Neo4j property graph. Findings, attack steps, credentials, assets and tamper-evident evidence receipts are typed nodes with pre/post-condition edges. The graph is wired to Claude Code or Codex through MCP with no custom harness. Five operator questions (next-step inference, entry-to-crown-jewel pathing, combined-severity promotion, evidence provenance, chokepoint identification) become reusable Cypher and Graph Data Science queries, plus a chain-severity score. On a synthetic multi-forest Active Directory lab modeled on GOAD, three findings rated Medium in isolation become Critical once their path to the forest root is materialized. Scoping each decision to a graph result cut the reasoning payload by about 98% on a token proxy. The authors say the results are illustrative single-run measurements, not a benchmark.

arxiv.orgpaperAdded 17 Oct 2026, published 14 Oct 2026🧠 AI

Trail of Bits tested 25 popular package managers with a Docker and mitmproxy harness that simulates a compromised package repository injecting malware into legitimate packages. Fewer than half enforce lockfile hashes by default, only two verified every download, and only Go verifies signatures by default. The post covers lockfile hash pitfalls (Pub, pnpm, Conan, CocoaPods, Composer), repository, developer and Sigstore signing, and Go's checksum database. It also proposes lockfiles that commit to a signing identity plus a transparency log.

blog.trailofbits.comwriteupPublished and added 14 Oct 2026🛡️ Security

OpenAI customer case study on Third Moment Research, which runs up to 10,000 concurrent agents on 36–48 hour autonomous quantitative-finance research sessions. Agents use the default Codex harness in a Dockerized environment with a terabyte of data. An outer harness driven by GPT-6 Sol or GPT-6.1 Sol watches for unfinished tasks and decides whether to retry, terminate or escalate to a human. Codex analyzed thousands of recorded trajectories and found a backtest bottleneck. A caching and memory-packing fix cut the backtest from about 3 hours to 3–5 minutes, which also reduced agent polling and context pollution. The post also covers how the team picks models by token efficiency and speedup benchmarks (GPT-6.1 Sol, GPT-6 Astra, GPT-5.6 Terra), and says Daybreak automates penetration tests of the sandbox.

openai.comarticlePublished and added 14 Oct 2026🧠 AI

AINews issue for 10/12–10/13/2026 covering the GA of Google's Gemini 4 Argon (pricing, Cursor/Devin/Copilot availability, CursorBench 52.8%, Artificial Analysis Intelligence Index parity with GPT-6 Astra at lower cost) and Meta's Muse Wonder, with independent evals and reliability caveats. It also summarises decision-model benchmarks (AA Decision Index, Jev 1.13, Nimble-v4), Fish Audio's open-weight Drama 3 TTS, agent-harness and agentic-RL research (Evo2, ReHarness, LongHarness, ThunderSync), Anthropic's move off Git, and safety items such as Argon's monitor evasion. It also notes AI Engineer NYC keynotes on AI x Finance. Useful as a curated, caveat-aware snapshot of the frontier-model landscape.

latent.spacearticlePublished and added 14 Oct 2026🧠 AI

AgentSWE is a benchmark of 25 tasks covering the agent development lifecycle: Creation of an agent from a natural-language requirement, Editing of production agent codebases, and Optimization toward a measurable target. Builders iterate on a few illustrative cases, then the frozen submission is graded on held-out cases with requirement-derived rubrics and hard pass/fail gates. Across coding agents, models and harnesses, none comes close to delivering the requirement. Recurring failures include overfitting to visible cases, hollow runtimes behind code that looks complete, and unsupported claims of completion.

huggingface.copaperAdded 13 Oct 2026🧠 AI