XBOW: Multi-View Embedding De-Duplication Cuts Finding Volume ~30%

What it is

An XBOW engineering post (by Adrian Losada, Offensive Security Engineer) on the de-duplication method behind their autonomous offensive security platform. The summary already states the outcome — four views, CWE-weighted recombination, ~30% volume reduction, 90% of duplicates eliminated. What the post adds is why the obvious approach fails and the specific noise sources you must control, which is the actionable part for anyone building similar finding-dedup.

How it works

The failure of the naive approach — embedding the whole finding as one text block — comes from two concrete noise sources the post names:

• Repeated boilerplate language. Phrases like "an attacker could" recur across unrelated vulnerability types (in descriptions, reproduction steps, and impact statements), pushing unrelated findings closer in vector space.
• Location string duplication. The same URL/path appears in the description, reproduction steps, and the location field, so it is effectively counted multiple times and inflates the similarity score.

The fix is to project each finding into four independent views (description, reproduction, location, impact), embed each separately, and recombine per-view cosine similarities as a weighted sum with weights that vary by CWE. This lets location be nearly irrelevant for an application-wide error disclosure (so occurrences across many URLs collapse into one canonical finding) but heavily weighted for XSS (so two reflected XSS in different parameters stay separate). One global similarity model cannot express both behaviors; per-CWE weights define a different "de-duplication topology" per class.

A second, easily missed step: cross-view pollution must be prevented. Early versions still leaked URLs/paths into the description view, defeating the purpose of down-weighting location. Each view must contain only its intended information.

flowchart TD
  F[Finding] --> D[Description view]
  F --> R[Reproduction view]
  F --> L[Location view]
  F --> I[Impact view]
  D & R & L & I --> W{CWE-specific<br/>weighted sum}
  W -->|error disclosure:<br/>low location weight| C1[Collapse across URLs]
  W -->|XSS:<br/>high location weight| C2[Keep per-parameter]

What to do

If you are building finding de-duplication: don't embed findings as one blob. Split into typed views, strip location strings out of every view except the location view, strip reusable explanatory boilerplate, and tune recombination weights per vulnerability class rather than globally.

Caveats

The post gives no weights, no embedding model, no CWE mapping, and no evaluation methodology — the ~30% and 90% figures are XBOW's own reported numbers on their own platform, not independently reproduced. The fetched text also cuts off mid-sentence during the cross-view-pollution section, so the full pollution-prevention technique is not captured here.