Lawyer with too many GPUs, part 5: DeepSeek-V4.1-Flash at near-NVFP4 quality on eight V100s from 2018. 524K context, 100+ tok/s decode, ~160 tok/s with two chats at once, and my legal harness now runs on it, 100% local
Reported by r/LocalLLaMA1 source ↓
Key points
- The author reports 95.1% next-token agreement with NVIDIA’s full NVFP4 pack on compared tokens
- The two-request test measured 159.0 tokens per second total; one request measured 118.9
- The setup keeps all routed experts in GPU memory during decoding, according to the author
- The legal harness is private; the author says drafts are reviewed before use and nothing leaves the building
Details
The author describes a new V100 configuration called Volta Max uPLIft, which keeps a 2-bit record of each routed expert and upgrades frequently used experts to NVFP4 weights. In the default setup, 5,949 of 15,360 experts are at NVFP4 precision. The author says roughly 78% to 85% of decode routes use those experts.
Benchmarks reported for eight V100-SXM2-32GB GPUs measured 118.9 tokens per second with one request and 159.0 total with two concurrent requests. At a 256,000-token prompt, prompt processing measured 2,329 to 2,430 tokens per second, with the first token arriving after 108.4 seconds. The system supports a reported context limit of 524,288 tokens per request.
Against NVIDIA’s full NVFP4 pack, the author reports 95.1% same-next-token results on compared tokens, versus 92.2% for the earlier 2-bit build. The author also reports lower KL divergence and fewer changed multiple-choice answers, while noting that the new build is slower than the 2-bit version in decoding and prompt processing.
The legal harness itself is not public. The author says it includes drafting tools, retrieval stores, tool calls and a router that can send requests to the V100 system or two DGX Sparks; all locally generated drafts are marked for review. The article says the setup is the author’s own project and that nothing is for sale.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh