Orca Security: AI Threat Detection

What it is

An Orca Security explainer on AI threat detection. The summary already carries its central argument (four techniques, four failure modes, telemetry-completeness thesis, the GuardDuty/Entra citations, the Orca mapping). What it omits is the operational content: a way to test a detection model, the specific questions to put to a vendor, and the concrete implementation practices — plus two numbers worth quoting.

What to do

The page gives a repeatable acceptance test for any detection model:

• Replay known-bad telemetry and count what fires. Atomic tests (one technique each) tell you whether coverage exists; a purple-team exercise that chains techniques tells you whether the model correlates them. Compare that coverage against what your threat-hunting team still finds by hand — "the gap is the honest number."

Vendor questions the page says to ask before buying:

• What telemetry does the model consume, and which entity does it profile?
• How long is the learning window, and what happens to an entity with no history?
• Can an analyst export the reason a finding fired? (The page calls this the one that matters most — a product that can't answer it produces alerts teams quietly stop reading.)

Implementation practices it prescribes:

• Fix collection before tuning — a model on partial telemetry is confident about a fraction of the estate, and no threshold change repairs that.
• Treat retraining cadence as a security decision; re-baseline after any incident that touched in-scope entities (the only reliable fix for baseline poisoning).
• Map every detection to a MITRE ATT&CK technique using its Detection Strategies vocabulary.
• Rank findings by exposure, not model score — a high-confidence anomaly on an isolated test box matters less than a medium-confidence one on an internet-facing role.
• Assign a response owner before the first alert; detection engineering owns thresholds/suppression/retraining, operations owns the containment call.

Preconditions / two numbers to hold

• Entra ID Protection learning period: earliest of 14 days or 10 logins — below that, an identity is scored against a population baseline or not at all (the cold-start gap on ephemeral workloads).
• Microsoft anomalous token detection was historically tuned to incur more noise than other detections; docs still report a higher-than-normal false-positive rate at low and medium risk even after tuning.

Caveats

Content-based classifiers need the payload, so encrypted traffic limits them; flow/DNS/control-plane logs suffice for anomaly and behavioral models. The page is vendor content and closes on Orca's Unified Data Model, SideScanning, Sensor, and Threat Investigation Agent — treat the framework as the transferable part, the product mapping as the pitch.