Show HN: I Turned DOOM into a Transformer

What it claims

Doomformer is an MIT-licensed Python toolkit. It compiles integer MLPs, spiking nets, transformers, Conv1d/Conv2d nets and in-map learners into ZDoom UDMF maps and runs them in ViZDoom 1.3.2. "EXACT" means DOOM's values equal an independent NumPy integer reference at tolerance 0. The author frames this as translation validation (Pnueli, Siegel and Singerman, 1998): a test of each compile on the inputs actually run, not a proof that the compiler is correct, and not a claim that the predicted digit is right. Prior DOOM-map computers (counter.wad, conway.wad, Danny Spencer's calculator) exist. What this project adds is the compiler, a measured list of engine behaviours that break exactness, and the zero-tolerance check.

How it works

Each doll is one unit of an activation. Each lava line a doll crosses adds a weight, a staircase is an activation, attention is a corridor of ceiling gates lowered by another live value, and ablation is a door. ACS scripts address cells, move input gates, rewind dolls and copy results out. They do none of the network arithmetic, with two exceptions: df.Agent takes its argmax in ACS, and at --opt 3 an ACS script copies each layer's light levels into the next layer's gate heights.

The compiler works around measured ZDoom 2.8.1 behaviours:

• A plane mover drops a second request while the first is still moving.
• A teleport line cancels lines crossed earlier in that tic.
• Things more than 32,767 units from a BSP node can be filed under the wrong sector, so maps fit in 32,000 units.
• Planes must start inside [-15,000, 16,000] and move within ±14,018. Fuzzing found that violations give wrong results, and check and compile now refuse them.

Large models use "tiles": one map per layer in resident games, with Python copying values between games.

Evidence

• Fuzzing. fuzz --edge ran 15,000 nets at range limits: 113,605 passes and 1,987,632 values, with no mismatch. A planted compiler bug is detected. Fuzzing found four real faults in the project's own maps (two scheduling bugs, two plane limits).
• Static extractor. It recovers the shipped MNIST net (784-64-45-10-10) from the WAD alone at --opt 0 to 3. It detects 17,906 of 23,622 single edits to a small map. Timing and geometry are reported as NOT CHECKED.
• Mutation testing. 297 of 1,004 mutation sites are killed. None of the 707 survivors is proven equivalent.
• MNIST throughput. Tiles read all 10,000 test digits in 1,390 s with 6 workers, at 216 tics per digit. One-map mode needs 5,696 tics per digit, or 272 at --opt 3. One-map mode read 1,000 digits in 2 hours on 14 games.
• Speed against NumPy (engine time). MNIST one-map is about 1.5 million times slower (73.2 s of engine CPU per pass). At --opt 3 it is 60,000 times slower (2.84 s), and in tiles about 15,000 times. Conveyor scrollers account for all of the --opt 3 cost. The author states that reaching 0.5 s per pass needs an engine change.
• Model quality.
• A 24k-parameter level-name transformer has perplexity 7.75, against 7.34 for its float stage and 5.84 for a 5-gram baseline.
• An induction transformer reaches 0.9995 held-out accuracy, against 0.233 for a 4-gram table.
• ask gets 94.7 % of training paraphrases right and 35.4 % of new wordings.
• The in-map perceptron reaches 71.2 %, against 81.0 % for an unclamped textbook perceptron.

How to use it

Requirements are Python 3.11+ on Linux or macOS. The first build needs git, cmake, a C compiler and network access to github.com.

pip install git+https://github.com/mottopanikeiku/doomformer   # Python 3.11+, Linux or macOS
doomformer hello             # 2 s: a small MLP, every value checked
doomformer draw --digit 7    # 15 s: the marine walks a 7 and the level reads it

doomformer doctor checks ViZDoom, Freedoom and the display. The Python API:

net = df.MLP(
    weights=[np.array([[1, 1], [1, 1]]), np.array([[1, 0], [0, 1]])],
    biases=[np.array([-1, 0]), np.array([0, 0])],
    thresholds=[np.array([1]), np.array([1])],
    signed_input=False,
)
with df.compile(net) as brain:
    for x in ([0, 0], [1, 0], [1, 1]):
        # computed by voodoo dolls; raises df.MismatchError on any difference from net.reference
        print(x, "->", brain(np.array(x)).tolist())
    brain.record([np.array([0, 1]), np.array([1, 1])], "dolls.gif")   # film the dolls doing it

df.MLP.from_torch quantises PyTorch models. It folds BatchNorm, lowers pooling, and takes GELU and SiLU as staircases. df.compile(net, opt=N) shortens the schedule, and doomformer plan predicts run time and recommends one map or tiles. df.run_many spreads inputs over local games or a Ray cluster you start yourself:

doomformer mnist --tiles --all --workers 6         # all 10,000 digits as tile maps, about 23 min
pip install "doomformer[ray]"
ray start --head                  # other machines: ray start --address=HEAD:6379
doomformer mnist --all --ray --record mnist.json   # all 10,000 digits, one map each, on a cluster

Budget one core and about 1.3 GB of RAM per game.

Limits

• Transformer conversion. Transformer.from_torch only converts the project's torch_models.GPT trained with its staircase quantisation recipe. It does not convert transformers trained elsewhere. No model has LayerNorm, and attention is integer attention.
• Engines. Only ViZDoom 1.3.2 is validated. No network larger than hello ran in other engines. GZDoom-family plane arithmetic differs. The compiler emits sector tags above 32,767, which Zandronum cannot find.
• Fuzzing gaps. The fuzzers do not cover learning maps, maps with more than 4,096 objects, or transformers in tile mode. ViZDoom lists at most 4,096 objects.
• Demos. Showcased seeds, names and lesions are picked. Marine policies imitate a scripted aimbot over 86 binned observations, with no RL and no learning from pixels. With demons awake, the marine dies before its first action lands (0 kills over 90 lives).
• Machines. The SUBLEQ and counter machines are finite-state, and the author makes no Turing-completeness claim.
• Licences. Code and docs are MIT. The level-name lists are CC BY-SA 4.0 and GPL-2.0. The MNIST samples are CC BY-SA 3.0. The FlyWire-derived fly circuits are CC BY-NC 4.0, which bars commercial use.