Renter reports near-linear Qwen inference scaling on AMD EPYC servers
Reported by r/LocalLLaMA1 source ↓
A LocalLLaMA community member benchmarked Qwen on rented AMD EPYC servers, reporting that throughput rose nearly linearly across concurrent streams with a 124k context window.
Key points
- The post reports 94 tokens per second at one stream and 456 at eight streams for Qwen 3.8 Flash Next Unsloth Q8
- The benchmark used two AMD EPYC 7642 processors and a ROME2D32GM-2T motherboard
- The author tested systems with 251GB and 504GB of DDR4 memory
- The author spent about $40 renting instances on Vast.ai for the tests
Details
The LocalLLaMA post reports that Qwen 3.8 Flash Next Unsloth Q8 reached 94 tokens per second at one concurrent stream and 456 tokens per second at eight. The author says performance scaled nearly linearly through at least eight streams with a 124k context window; they did not test more than eight.
The tests used two AMD EPYC 7642 processors on a ROME2D32GM-2T motherboard, with configurations containing 251GB and 504GB of DDR4 memory. The author says the benchmarking cost about $40 and also tested other models and runtimes, including Maya with GLM 5.3F and DSV4.1F. These are results reported by the post author, not an independent evaluation.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh