Weighted State Souping for Efficient Long-Context Retrieval
Reported by Apple Machine Learning Research1 source ↓
Apple researchers report a method that learns query-dependent weights to combine cached state space model encodings. The paper was accepted at the PALM workshop at NeurIPS 2026.
Key points
- The method composes cached encodings with weights that depend on the query
- The authors report gains on document question answering, retrieval, variable tracking and tool use
- A sparse version is reported to have near-constant encoding cost as context length grows
Sources
Details
The paper introduces weighted state souping, which combines independently encoded chunks such as documents, tool definitions or memory records. Unlike standard state souping, the proposed method learns how much each cached state should contribute for a given query.
The authors report improved performance on single- and multi-hop document question answering, needle-in-a-haystack retrieval, variable tracking and tool use. They also describe a sparse version with near-constant encoding cost as context length increases, which they say preserves or sometimes improves performance.
The paper was accepted at the Workshop on Personalized, Aligned, Long-term Memory for AI Systems at NeurIPS 2026. The supplied article does not provide benchmark scores or details about the models and evaluation datasets.
Written from coverage as of
Images
- Apple Machine Learning Research
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh