Orca Security: AI Security Tools: How to Evaluate Them Across Every ML Attack Phase

What it is

An Orca Security vendor post arguing that six AI security tools each cover only one ML attack phase and none cover cloud infrastructure misconfiguration around AI workloads. The summary already carries the tool list, the five-phase mapping, the eight criteria, and Orca's positioning. What it omits — and the only reason to write this — is the concrete, checkable misconfiguration pattern the page names at the field-value level.

The attack chain worth checking

The page repeatedly grounds its "individual findings look medium, chained they're critical" argument in one specific AWS triad that forms a credential-theft path:

• Overprivileged SageMaker execution role attached to a training job
• Training data S3 bucket with BlockPublicAcls: false
• EC2 training instance without HttpTokens: required (IMDSv2 not enforced)

The mechanism: an attacker who compromises the training instance, with IMDSv2 unenforced, reads the EC2 instance-profile credentials from the metadata service, then uses the overprivileged execution role to reach any S3 bucket or RDS instance that role can access. No single finding here rates critical in isolation.

flowchart LR
  A[Compromised training instance] -->|HttpTokens not required| B[IMDSv2 unenforced]
  B --> C[Steal instance-profile creds]
  C -->|overprivileged exec role| D[Account-wide S3 / RDS read]
  E[Bucket BlockPublicAcls: false] -.exposed training data.-> A

Concrete control references

Beyond the summary's framework names, the page cites specific controls an engineer can map against:

• CIS AWS Foundations Benchmark v3.0 control 1.16 — the IAM-policy check for the SageMaker execution role
• IMDSv2 enforcement per CISA cloud security technical reference architecture (Sept 2023)
• NIST SP 800-218A (SSDF for AI/ML, draft 2024) references adversarial robustness testing as a required practice for high-risk systems — the compliance hook for ART
• ATLAS technique IDs behind the phase mapping: reconnaissance AML.T0037, initial access AML.T0012, model manipulation AML.T0031, data poisoning AML.T0020, post-exploitation AML.T0025; Garak covers AML.T0051 (prompt injection) and AML.T0054 (jailbreak); Viper models ATT&CK T1021 and T1550

Caveats

This is Orca marketing; the "gap" it identifies is precisely the one its agentless SideScanning product fills. The 40% shadow-AI figure is attributed to a "2024 Gartner AI TRiSM market survey" with no link. Tool coverage claims (ART's 39 attack / 29 defense modules) are stated, not benchmarked. Treat the config triad as a useful self-audit checklist, not as validation of the vendor's completeness claim.