Joined the 4x R9700 Club | Initial Benchmarks for QFN 3.8 Below
Reported by r/LocalLLaMA1 source ↓
A LocalLLaMA user shared early benchmarks from a four-card Radeon AI PRO R9700 inference server running Qwen3.8-Flash-Next. The tests measured performance across 1 to 8 users and contexts up to 128K tokens.
Key points
- The user reports about 6,800 tokens per second of prefill throughput across concurrent users
- Eight simultaneous 128K-token prompts took about 151 seconds to reach the first token
- Cached follow-ups reached 80 to 225 tokens per second per user across the tested loads
- The four-card server cost roughly C$16,000, according to its owner
- The owner says the results are initial benchmarks, with further testing planned
Details
The r/LocalLLaMA user reports testing Qwen3.8-Flash-Next on four Radeon AI PRO R9700 cards using Radiance. The tests covered contexts from 32K to 128K tokens and between one and eight concurrent users; the owner described the results as initial benchmarks.
In the reported worst-case burst, eight users each submitted a fresh 128K-token prompt at once. The first-token time was about 151 seconds, while the owner estimated prefill throughput at roughly 6,800 tokens per second regardless of user count.
For follow-up turns, which reused cached conversation context, the user measured first-token delays of about 0.2 to 2.5 seconds and output rates of 80 to 225 tokens per second per user. The owner says 94% to 99% of those follow-up prompts came from cache.
The server uses an EPYC 7402P processor, 128 GB of RAM and four R9700 cards limited to 210 watts each. The owner reports spending roughly C$16,000 on the system and says it is intended for multi-user inference at their business.
Written from coverage as of