64GB isn't the dead zone anymore if you run agents
Reported by r/LocalLLaMA1 source ↓
Key points
- The setup uses Qwen3.8 27B in LM Studio with a 256k context
- At 164,000 tokens, the engine used 48GB, including 46GB of GPU memory
- A prompt-cache hit rate of 93% reduces each turn to about 1,000 new tokens
- A full prefill after a cache miss takes about 20 minutes
- The poster says 48GB is insufficient at 256k context with this setup
Details
The report describes ten days of agentic coding sessions on a 64GB M4 Max. The poster says the model weights take 16GB and the KV cache about 11GB at 164,000 tokens; additional memory is attributed to context caching and MLX scratch memory.
The poster says the prompt cache is essential to keeping sessions usable: at a 93% hit rate, a turn processes about 1,000 new tokens rather than 160,000. When the cache misses, a full prefill takes about 20 minutes.
According to the post, LM Studio's MLX vision path does not support KV-cache quantization for this model. The text-only path does, but the poster says it skips cache checkpoints when quantized, causing more reprocessing; they suggest reducing context to about 128k on 48GB or using llama.cpp with q8 KV, estimated at 25–30GB for 256k context but with slower prefill.
Written from coverage as of