Are bespoke inference engines overhyped? My Strata experience on 2× R9700
, updated First seen Updated Limited coverage, tapering
Industry
Reported by r/LocalLLaMA1 source ↓
A LocalLLaMA user says Strata ran Qwen Next Flash at about 60 tokens per second on two R9700 cards, but was stable only with experts in VRAM and delivered a lower HumanEval+ score than their llama.cpp setup.
Key points
- The user reports Strata hung on long tasks when experts were offloaded to RAM
- Keeping the setup stable required fitting all experts in 64 GB of VRAM, which limited them to a 3-bit quant
- The 3-bit model scored 93.9% on HumanEval+, versus 95.1% for Qwen3.8-27B Q8 on llama.cpp
Details
The user, posting on r/LocalLLaMA, tested Strata with Qwen Next Flash on two R9700 cards.
They report that configurations with experts offloaded to RAM hung during long tasks despite trying different settings. The setup was stable when all experts fit in VRAM; with 64 GB, that meant using a 3-bit quant. The user reported roughly 60 tokens per second, faster than llama.cpp, but said the Strata model needed nearly twice as many tokens to reach its HumanEval+ result.
Written from coverage as of