Splitting up work to be split across mutiple slots on the same gpu for higher total tokens per second
Reported by r/LocalLLaMA1 source ↓
A LocalLLaMA user reports higher combined token throughput when running multiple Qwen 3.8 27B slots on a 7900 XTX with hipfire. They ask how to preserve context capacity and divide tasks that depend on one another.
Key points
- The user says adding slots reduces the context available to each slot
- They are exploring shared context or an orchestration slot to divide prompts into parallel tasks
- The setup also includes a 6800, a 6600 and 96 GiB of RAM for the CPU
Details
The user says running multiple slots at once produced significantly higher total tokens per second for Qwen 3.8 27B on a 7900 XTX using hipfire. The post does not give measured throughput figures or compare specific slot counts.
The user identifies shrinking context per slot as one obstacle and asks whether memory can be used more efficiently. They also note that some tasks are difficult to split into parallel parts without information about earlier or later steps.
Ideas raised in the post include shared context or a messaging board, and an orchestration slot that divides a prompt among other slots before stopping to free VRAM. The user also suggests routing tasks through a decision model, but reports no results from these approaches.
Written from coverage as of