Skip to content

Back to the catalog

dfir.chwriteup

Reptile's Custom Kernel-Module Launcher

A deep-dive analysis of the Reptile Linux rootkit's custom kernel-module launcher, which replaces insmod with a direct init_module syscall to load an XOR/rotate-encrypted kernel object compiled with a random per-build key. The author walks through the Makefile encryption step, the do_decrypt/custom_rol32 obfuscation routine, and the raw syscall macro that bypasses standard command-line loading, then shows why insmod-based detections (like a common Elastic rule) miss it and how auditd_manager rules watching init_module/finit_module syscalls (Elastic's Kernel Driver Load rule) can still catch it.

TL;DR Reverse-engineers how the Reptile rootkit's custom loader encrypts and injects its kernel module via a raw init_module syscall to evade insmod-based detections, and shows the auditd-based syscall rule that still catches it.
Open original ↗
Document

What it is

A technical breakdown of the custom userland loader (loader.c) shipped with Reptile rootkit v2.0, which UNC3886 used in the espionage operations Mandiant documented in Cloaked and Covert (2024). The key point for a defender: this loader deliberately abandons insmod, so detections keyed on that command name miss it entirely.

How it works

  • At compile time the Makefile generates a random 32-bit key (RAND2) and XOR-encrypts the compiled reptile.ko into reptile.ko.inc. The random per-build key defeats hash- and hex-string hunts for the module on disk.
  • The loader embeds that encrypted blob as a static array (reptile_blob), decrypts it in memory, mallocs a buffer, copies it in, and loads it.
  • Encryption/decryption is one function (do_encode/do_decrypt): a per-word XOR against custom_rol32(key ^ len, len % 13), walking the buffer 4 bytes at a time. custom_rol32 is a standard rotate-left.
  • Loading bypasses insmod by calling the syscall directly via a macro:
#define init_module(module_image, len, param_values) syscall(__NR_init_module, module_image, len, param_values)

That direct-syscall path is why command-name monitoring fails but syscall auditing still catches it.

What to do

The insmod process rule the loader evades:

process where host.os.type == "linux"
and event.type == "start"
and process.name == "insmod"
and process.args : "*.ko"

Instead, monitor the syscalls the loader actually uses via auditd_manager (configuration per Elastic's Kernel Driver Load rule page):

driver where host.os.type == "linux"
and event.action == "loaded-kernel-module"
and auditd.data.syscall in ("init_module", "finit_module")

Also watch for a tainted kernel as a secondary signal that an unexpected module was loaded.

Caveats

The author states syscall-level auditing "might be enough" to catch the load — it is presented as a heads-up signal, not a guaranteed block, and depends on auditd being configured to capture init_module/finit_module. The disk-hunting evasion (random key) is not addressed by these detections at all.

Technologies named here

Techniques named here

  • Encrypted kernel module payload
  • Custom XOR/ROL32 decryption routine
  • Direct syscall invocation to bypass command-line monitoring

Threat actors named here

Related in Malware & RE