Stop counting tokens, start counting what ships
Reported by MIT Technology Review1 source ↓
The article argues that companies should track AI coding costs against merged pull requests, not just token use. It recommends routing routine work to lower-cost or owned models while reserving premium models for harder tasks.
Key points
- The proposed core metric is AI-development cost per merged pull request, tracked alongside rework and defects
- The article recommends routing tasks by complexity, sensitivity, cost and latency
- HPE reports that its top 5% of users generated most consumption, with the heaviest using about 450 times more than the lightest
Details
The article says coding agents can use 10 to 50 times more tokens on a task than a basic chat prompt because they plan, generate code, run tests and repeat steps. It argues that token costs alone do not show whether that spending produces useful software, and proposes measuring total AI-assisted development cost per merged pull request while also tracking rework and defects.
It recommends using frontier models for difficult planning and novel reasoning, and considering open-weight models for routine coding, testing and refactoring after evaluating them on an organization’s own code and acceptance criteria. The article also suggests comparing rented inference with owned capacity, accounting for power, operations, maintenance and depreciation; it says low or unpredictable usage may favor APIs, while sustained demand may support ownership.
HPE says a hybrid platform routes workloads from more than 150 internal applications to on-premises open-weight models and has reduced token costs by 30% to 50%. HPE also reports that its top 5% of users generated most consumption, and that the heaviest users consumed roughly 450 times as much as the lightest users on the same license.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh