Skip to content

Back to the catalog

Ned Peterson (@NedPeterson99) on X
x.comthread

Ned Peterson (@NedPeterson99) on X

Social post summarizing a Sola Security study (Leon Goldberg, Gal Engelberg). An AI agent re-scored 9,967 high-severity findings from two commercial cloud security platforms across eight production environments and changed severity on 75.1% (58.5% downgraded, 16.6% upgraded). The cause is that 599 vendor rules each carry one fixed severity regardless of environment, and 49% of verdicts needed context beyond the flagged resource. Caveats: 92% of findings were AWS, and the study did not test whether the new severities predict real exploitation.

TL;DR AI agent re-graded ~10k "high" CNAPP/CSPM findings and changed severity on 75%, mostly downgrades, because vendor rules ignore environmental context.
Document

What it is

A social post relaying a Help Net Security write-up of a Sola Security study by Leon Goldberg and Gal Engelberg. The post is a secondary source, and the study itself was not fetched. An agent re-scored 9,967 "high" findings from two commercial cloud security platforms (CNAPP/CSPM) across eight production environments.

How it works

Each of the 599 vendor rules carries one fixed severity regardless of where the resource lives. The agent re-scored findings using environmental context, and 49% of its verdicts depended on evidence beyond the flagged resource, such as connected assets or data from other security tools.

Figures the summary omits:

  • The most common downgrade, at 32.3% of downgrades, was an inert resource attached to nothing.
  • Read-only probes of eight AWS accounts matched the agent's facts 99.4% of the time, where those facts could be checked.

What to do

The post's advice to cloud and security teams:

  • Ask your CNAPP/CSPM vendors how much context, reachability and cross-tool evidence actually feeds the severity score.
  • Take a sample of your own "high" findings and measure your downgrade rate. The study's 58.5% downgrade and 16.6% upgrade rates are the benchmarks to compare against.
  • Put the triage hours saved into the findings that turn out worse than the dashboard says, since 16.6% were upgraded.

Caveats

  • 92% of findings were AWS, so the results may not carry over to Azure or GCP.
  • The study did not test whether the revised severities predict real exploitation. A lower severity is therefore not shown to be safe to ignore.
  • The 99.4% verification applies only to facts that could be checked, not to the severity judgments themselves.
  • The post does not name the two vendors or describe the agent's method, so the sampling procedure cannot be reproduced from this source.

Organisations named here

  • Gal Engelberg
  • Help Net Security
  • Leon Goldberg
  • Sola Security

Technologies named here

  • Read-only probes of eight AWS accounts
  • AWS
  • CNAPP
  • Cloud Security Posture Management

Techniques named here

  • Downgrading findings on inert resources attached to nothing