Crossing the Threshold: How OpenAI and Anthropic Approach Autonomy and Safety in Financial Agents

What it claims

This is a conference recap, not a technical write-up. Its shared thesis is that safe agent autonomy depends on where you place controls and human judgment, not on how many restrictions you add. Beyond the catalog summary, the page adds a few concrete points:

• Approval fatigue is a risk in itself. Feminella (Two Sigma) says a human who approves everything slows teams down. That person also stops paying attention, so approving everything creates risk instead of removing it.
• Rules versus classifiers. Use deterministic rules for clear-cut cases and classifiers for ambiguous ones. Auto-mode classifiers handle routine calls and send humans only the requests "most likely to make a difference."
• Propose, don't act. Shahab (Anthropic) uses a trade-break investigation agent as the example. It may propose amendments to orders data but must not write them. A human approves each amendment.
• A real prompt-injection failure. Rybka (Bloomberg) describes an agent that treated instructions hidden in a marketplace review as an order to stock a product. His mitigations:
• input validation
• prompt-injection defenses
• provenance tracking
• human approval for high-value actions

His full control list also includes outbound traffic restrictions, which the summary leaves out.

• Traceability. Kim (OpenAI) says finance users must be able to see where a number came from. ChatGPT for Financial Services builds that review step into the product.

How it works

The routing pattern Feminella describes:

flowchart LR
  A[Agent request] --> B{Clear case?}
  B -->|yes| C[Deterministic rule]
  B -->|no| D[Classifier]
  D -->|routine| E[Auto-approve]
  D -->|consequential| F[Human review]

Rybka's principle sits around this routing. Every call is checked and logged outside the model, and a kill switch stays available.

Limits

• No evidence. The page gives no metrics, false-positive rates, cost figures or incident counts. Every claim is a speaker's assertion.
• Incomplete control list. Two Sigma has "five controls," but the page names only one: auto-mode classifiers.
• No implementation detail. Nothing is said about how the classifiers are built or trained.
• Predictions, not findings. Kim's 2027 predictions about trading agents and less reliance on Excel and PowerPoint are forecasts.
• Product mentions without detail. The page mentions Claude for Financial Advisors, announced the month before, but gives no technical detail about it.
• Sponsored content. The page includes a sponsor block for shop.dev, which is unrelated to the topic.

The page is useful as a short checklist of control patterns and one concrete injection failure. It is not a design reference. For implementation detail, look for the talks themselves or the promised podcast with Kim.