Developer fine-tunes a decision model to play Doom on a Mac
Reported by r/LocalLLaMA1 source ↓
A developer trained an image-based decision model to play ViZDoom’s Defend the Center arena on Apple Silicon, using labeled game frames and LoRA fine-tuning.
Key points
- The model averaged 23.2 kills across 20 episodes, according to the developer
- The developer says it averaged 19.7 kills on Freedoom, whose monster sprites were not in training
- The open-source tool is free and requires Apple Silicon, macOS 14 or later, and Python 3.12 or later
Details
The developer reports fine-tuning d1-2B to react to game images in ViZDoom’s Defend the Center arena. The model answers questions about whether a shot would hit, where the nearest monster is, and whether one is dangerously close; a Python loop uses those answers to turn and shoot.
Training used about 3,000 frames labeled from the game engine’s monster positions. The developer says fitting a readout took seven minutes, followed by 43 minutes of LoRA fine-tuning over two epochs on an M5 Pro, with the model’s vision tower frozen.
On 510 held-out frames, reported precision for whether a shot would hit rose from 0.79 to 0.92; nearest-monster location accuracy went from 92.7% to 96.8%, and dangerously-close accuracy from 93.9% to 98.3%. The developer reports an average of 23.2 kills over 20 episodes and 19.7 on Freedoom; the setup is an arena where the player can turn but not move through a full Doom level.
The developer says the project is free, MIT-licensed, and available through the mlx-decision tool for Apple Silicon. The post lists macOS 14 or later and Python 3.12 or later as requirements.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh