Qwen3.5-9B running on a PS4’s GPU, offline, using its own MTP head for speculative decoding
Reported by r/LocalLLaMA and Tom's Hardware3 sources ↓
Developer Leonardo Alegría’s open-source project runs Qwen3.5-9B offline on a jailbroken PS4 at 31–39 tokens per second, using the model’s MTP head to draft tokens for speculative decoding.
Key points
- The model’s MTP head drafts up to two tokens for the full model to verify
- Weights are compressed to about 4.5 GiB using custom 4-bit and 3-bit formats
- Tom’s Hardware reports code generation reaches up to 43 tokens per second
- The free 5.66GB project requires a jailbroken PS4 running GoldHEN
Sources
- r/LocalLLaMARead ↗Qwen3.5-9B running on a PS4’s GPU, offline, using its own MTP head for speculative decoding
- r/LocalLLaMARead ↗[Paper & Models] MMFlash: Faster Speculative Decoding for Vision-Language Models
- Tom's HardwareRead ↗Jailbroken PS4 runs 9-billion-parameter AI model at up to 43 tokens per second
Details
Tom’s Hardware reports that Garlic LLM runs Alibaba’s Qwen3.5-9B on the PS4’s GPU, with the app building low-level GPU commands itself. The developer tested it on a base or Slim PS4 running firmware 11.52; the project does not support PS5.
The model weights use a mix of 4-bit and 3-bit formats to fit in about 4.5 GiB. With speech models included, the project uses about 5.23 GiB, and its attention cache is 4-bit, supporting a 16K-token context rather than the model’s native 256K.
The developer says using the model’s multi-token prediction head to draft up to two tokens raised text-chat speed from 22 to 31–39 tokens per second.
Garlic LLM is a free, 5.66GB download available through Hugging Face and installable with the PS4 Package Installer. The app also supports voice replies in seven languages and dictation in 25 European languages.
Written from coverage as of
Images
- Tom's Hardware
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh