A LocalLLaMA post comparing data-center cards across Kimi K3 and three other models reports that AMD’s MI455x performed best. It says larger batches increase total batch throughput while reducing tokens per second for each request.
Key points
The post reports AMD’s MI455x came out on top in the comparison
Larger batches raised total batch throughput but lowered per-request speed
The comparison used Kimi K3 and three other models
The LocalLLaMA post compares the performance of large data-center cards on Kimi K3 and three other models. It says the AMD MI455x came out on top, but provides no scores or further benchmark details in the supplied text.
The post describes a tradeoff with larger batches: the batch processes more tokens per second overall, while each individual request receives tokens more slowly.