Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium
Reported by Hacker News1 source ↓
TrnRT is an open-source inference engine for serving large language models on AWS Trainium. Its first supported model is GLM-5.3-Flash.
Key points
- TrnRT serves an OpenAI-compatible API on AWS Trainium
- GLM-5.3-Flash is the first supported model
- Serving requires a trn2.48xlarge; compilation can run on an x86_64 host
Details
The project’s Hacker News post describes TrnRT as an inference engine for AWS Trainium that uses NKI megakernels for decoding and a prefix cache spanning device and host memory. The post says the engine is designed to support additional model families, but does not name them.
The post reports a SemiAnalysis AgentX test on one trn2.48xlarge, using a 1-million-token context and 32 concurrent agents for an hour per test point. It cites a spot price of $8.60 per hour in us-east-2 on Nov. 3, 2026, and says batch 12 cost $0.167 per million tokens on a $35.76-per-hour capacity block. The project notes these results were measured with an earlier TrnRT build.
Written from coverage as of
Images
- Hacker News
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh