Crossing the Threshold: How OpenAI and Anthropic Approach Autonomy and Safety in Financial Agents

What it is

A recap of AI Engineer New York talks on agent autonomy in finance, from OpenAI, Anthropic, Two Sigma and Bloomberg. The page is a conference summary. It carries no code, configuration or benchmarks, so the value is in a few design rules.

How it works

The speakers converge on one placement rule: controls sit outside the model, and human judgment is spent only where it matters.

flowchart LR
    A[Agent action] --> B{Clear case?}
    B -- yes --> C[Deterministic rule]
    B -- ambiguous --> D[Classifier]
    D -- routine --> E[Auto-allow]
    D -- important --> F[Human approval]
    C --> G[Log every call]
    E --> G
    F --> G

• Rule vs classifier (Two Sigma). Deterministic rules handle clear cases and classifiers handle ambiguous ones. The classifier decides routine calls and routes only the most consequential requests to a person. The aim is to avoid approval fatigue, which the speaker said also adds risk because reviewers stop paying attention. Each control is meant to do one job and compose with the others.
• Injection example and fixes (Bloomberg). An agent treated instructions embedded in a marketplace review as an instruction to stock a product. Rybka's listed mitigations are input validation, prompt-injection defenses, provenance tracking, and human approval for high-value actions. The general rule given was "Check every call, log it, and keep a kill switch."
• Propose, don't act (Anthropic). A trade-break investigation agent recommends an order amendment and a human approves it. The agent does not write to the orders data. The trust layer's third part is re-checking that the agent still works after variables change over time.
• Traceability (OpenAI). In finance you must be able to check where a number came from. OpenAI built that review into the ChatGPT for Financial Services experience. Which steps a human controls, reviews or approves is framed as a product design question.

What to do

• Give agents read or propose access by default, and gate writes behind approval.
• Gate by value or consequence, not by volume, so reviewers are not approving everything.
• Track provenance of any text the agent ingests, such as reviews, since that text can carry instructions.
• Re-run regression checks whenever the model, prompts or tools change.

Caveats

• Two Sigma's talk covered five controls, but the page names only the auto-mode classifier.
• OpenAI's 2027 claim that agents will do "investment idea generation and trading" is a prediction, not a result.
• The page gives no thresholds, tooling or implementation detail for any control.