Nvidia researchers argue huge AI context windows are a dead end
Reported by The Decoder1 source ↓
Nvidia researchers say language models often do not need to process all available context at once. In many tests, periodic summaries and targeted lookups performed about as well.
Details
The Decoder reports that the Nvidia team considers very large context windows often excessive and costly. Its researchers found that periodic summaries and targeted lookups worked about as well in many tests, though the article gives no test details or specific measurements.
The article describes a context window as the information a model holds in view while generating a reply. It says GPT-4 launched in March 2023 with 8,192 tokens, while GPT 5.6 Sol can handle around one million tokens.
Written from coverage as of
Images
- The Decoder
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh