The Iron(wood) Throne: Google's TPU Stakes Its Claim on Inference
Reported by SemiAnalysis1 source ↓
SemiAnalysis reports benchmark results showing Google’s TPUv7 is competitive with, or cheaper than, Nvidia and AMD accelerators on several long-context, multi-turn inference workloads. Results vary by model, hardware and precision.
Key points
- SemiAnalysis reports TPUv7 matched or beat competing accelerators on cost efficiency across four model benchmarks
- For GLM-5.2, TPUv7 delivered 1.2x to 2.7x the tokens per dollar of B200, B300 and MI355X at tested speeds
- For Qwen3.5, TPUv7 beat GB300 FP8 at every tested interactivity level using llm-d disaggregated serving
- The results were measured on InferenceX’s AgentX benchmark for long-context, multi-turn inference
Details
SemiAnalysis says the results are the first official AgentX benchmark results for TPUv7, covering Muse Spark Open, GLM-5.2, Qwen3.5 and Kimi K3. The tests use different serving systems and weight formats, so comparisons are specific to each model and configuration.
For GLM-5.2, SemiAnalysis reports TPUv7 was the cheapest FP8 option, delivering 1.2 to 2.7 times the tokens per dollar of B200, B300 and MI355X across tested serving speeds. It says Nvidia’s NVFP4 configuration was faster on cost efficiency, but at a loss in accuracy or quality. For Qwen3.5, TPUv7 beat GB300 FP8 at every tested interactivity level and reached parity with GB300 FP4 at higher quality, using llm-d disaggregated serving.
For Kimi K3, SemiAnalysis reports TPUv7 was competitive with GB300 NVL72 and delivered 1.15 to 2.5 times the tokens per dollar of GB200, B200, B300 and MI355X. It describes the Muse Spark Open result as competitive on performance per dollar, noting that the model architecture was co-designed for GPUs. The article does not provide detailed numerical results for that comparison.
SemiAnalysis says the benchmark was run on InferenceX/AgentX, which it describes as a widely supported inference benchmark. It also reports that Nvidia submitted a GB300 FP4 counter-result for Qwen3.5, which remained 1.64 times worse per dollar than TPUv7 FP8 across most of the tested curve; the article’s comparisons are the publisher’s reported results.
Written from coverage as of
Images
- SemiAnalysis
- SemiAnalysis
- SemiAnalysis
- SemiAnalysis
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh