Skip to content

Back to the catalog

Inside Microsoft Defender's Signature World
retooling.iowriteup

Inside Microsoft Defender's Signature World

RETooling researchers reverse-engineer Microsoft Defender Antivirus (MDA), detailing the .vdm signature database format, the MpEngine.dll module loading/signal-dispatch process, and the internal structure of PEHSTR and PEHSTR_EXT byte/string-matching signatures used to detect PE malware. The post decodes the wildcard opcode family (0x90 0x01–0x05) used in Defender signatures and shows Yara-equivalent translations plus a worked example bypassing a Plugx.C detection by modifying a single matched byte, framing the work as groundwork for adversary emulation and signature evasion.

TL;DR Deep-dive RE of Microsoft Defender's .vdm signature format and PEHSTR/PEHSTR_EXT wildcard pattern language, with Yara translations and a real detection-bypass example.
Open original ↗
Document

What it is

RETooling's reverse-engineering of Microsoft Defender Antivirus (MDA) signatures, analyzed on product version 1.1.23100 (Oct 2023) / signature version 1.401.1166.0. Beyond the format overview, the page gives reproducible detail an engineer can act on: where signatures live, how to extract them, the weight/threshold matching mechanism, and the exact semantics of each wildcard opcode.

How it works — the parts you can reproduce

Extraction. Signatures ship in four .vdm files under C:\ProgramData\Microsoft\Windows Defender\Definition Updates\<RandomGUID>\: mpavbase.vdm/mpasbase.vdm (monthly base, anti-malware/anti-spyware) and mpavdlta.vdm/mpasdlta.vdm (daily deltas merged in memory at load). The base files are PE files holding a resource whose header starts with magic RMDX. The payload (at offset 0x138 in the example) is raw-deflate compressed:

import zlib
compressed = open('x.gz', 'rb').read()
decompressed = zlib.decompress(compressed, -zlib.MAX_WBITS)

Match mechanism. Each PEHSTR/PEHSTR_EXT threat is a set of weighted sub-rules bracketed by SIGNATURE_TYPE_THREAT_BEGIN (0x5C) / THREAT_END (0x5D). Detection fires when the summed weights of matched sub-rules meet the header threshold. Worked example: threat Darby.A needs threshold 0x33; five sub-rules weigh 0x0A each, four weigh 0x01 — so all five heavy rules plus at least one light rule triggers it. Missing a single byte of any required sub-rule breaks the match. This is the lever behind the single-byte evasion.

Wildcard opcodes. All wildcards begin 0x90; the second byte selects behavior (range 90 01–90 20). The five decoded here, with their Yara equivalents:

OpcodeMeaningYara
90 01 XXjump/match XX bytes of any value?? ?? … (XX wildcards)
90 02 XXmatch up to XX bytes at this position[0-XX]
90 03 XX YYalternation: match one of two sequences of length XX or YY`(seqA\
90 04 XX YYregex-like, XX = exact byte count, YY = pattern length[30-39] etc.
90 05 XX YYlike 90 04 but XX is an upper bound and matching is case-insensitive[a-zA-Z]{0,N}

What to do

Verify a signature match with the built-in tool (used in the post to confirm Darby.A detection and its byte-modified bypass):

MpCmdRun.exe

For emulation/evasion work: extract with the snippet above, locate the threat's sub-rules and threshold, and modify any single required (non-wildcarded) byte to drop below threshold — or craft an artifact that satisfies it.

Caveats

The authors state plainly that byte/string matching — even with wildcards — is "relatively easy to bypass," and that proper adversary emulation "goes beyond a pattern based detection." Several struct fields (ui16Unknown, ui8CodeUnknown, multiple unknownBytes) remain unidentified. Dynamic HSTR signatures tied to MAPS cloud-delivered protection were noticed but not investigated, so on-host .vdm analysis does not account for cloud-side detection.

Organisations named here

  • Recon 2024
  • RETooling

Technologies named here

  • Microsoft Defender Antivirus
  • MsMpEng.exe
  • MpEngine.dll
  • Microsoft Defender for Endpoint
  • Windows
  • WdBoot.sys
  • WdFilter.sys
  • MpSvc
  • MsRtp
  • MpCmdRun
  • MpClient
  • YARA
  • zlib
  • Custom Python VDM parser script
  • Windows Defender

Techniques named here

  • Signature format reverse engineering
  • Byte/string pattern-based malware detection
  • Wildcard pattern matching in signatures
  • Signature-based detection evasion/bypass
  • Adversary emulation
  • ELAM early-boot driver integrity scanning

Threat actors named here

  • Plugx.C
  • Darby.A
  • Plugx.A
  • Banker.YB
  • Stration.CC

Related in Malware & RE