QEngine 1.3.1 leads tested engines in Qwen3.8-Flash-Next decode on two RTX 5090s
Reported by r/LocalLLaMA2 sources ↓
A LocalLLaMA user reports QEngine 1.3.1 reached up to 504 tokens per second decoding Qwen3.8-Flash-Next on two RTX 5090 GPUs. The user says results benefited from repeated prompts and an adaptive expert cache, so they may not represent varied traffic.
Key points
- QEngine decoded at up to 504 tokens per second in the user's repeated-prompt benchmark
- On varied traffic, the user measured about 300–400 tokens per second
- QEngine and Strata had similar prompt-read latency in a 12-turn agent-use test
- RAM-based KV caching kept all eight tested idle conversations warm after other traffic
- The user had to compile QEngine for RTX 5090s by changing its CUDA architecture setting
Sources
- r/LocalLLaMARead ↗Qwen3.8-Flash-Next on 2x RTX 5090 with 504 tok/s decode: QEngine 1.3.1 decodes ~1.5-1.9x faster than the next-best engine I've tried (numbers, caveats, and what still bites)
- r/LocalLLaMARead ↗Getting incredible results with a single RTX 3090ti and 128gb of ram. Deepseek-V4.1-Flash-1017 (35 tok/s decode) and Qwen3.8-Flash-Next (130 tok/s decode)
Details
The LocalLLaMA poster tested Qwen3.8-Flash-Next on a system with two RTX 5090 GPUs, a Threadripper PRO 3975WX processor and 251 GB of RAM. QEngine 1.3.1 decoded at 370 tokens per second for narrative prompts and 439 for code with its default VRAM cache; with KV caching in RAM, those rates were 406 and 504. The poster compiled the software for RTX 5090s by changing its CUDA architecture setting from sm_86 to sm_120.
The poster cautioned that the benchmark repeated prompts, which favored QEngine's adaptive expert cache. They reported about 300–400 tokens per second on varied traffic. In the same benchmark, Strata decoded at 180–293 tokens per second across two quantization settings, while tuned vLLM reached 135–141.
In a 12-turn tool-use test with context growing to 35,000 tokens, QEngine took 1.75 seconds to read the final prompt, compared with 1.7–1.9 seconds for Strata. With RAM caching, all eight tested conversations of about 58,000 tokens resumed warm after 400,000 tokens of other traffic; with VRAM caching, only two resumed warm, while the other six took 8.7–9.4 seconds to re-prefill.
The poster reported 44 of 44 tasks passed on QEngine 1.3.1 in one assistant-harness run, but described the one-task difference from version 1.3 as noise. In a coding test run on an earlier build, QEngine scored 26 of 30 and Strata configurations scored 24–25; the poster said the results did not establish a quality advantage.
Written from coverage as of