Lawyer’s eight-V100 AI server moves DeepSeek from 2-bit to near-NVFP4 precision
Reported by Tom's Hardware1 source ↓
A lawyer upgraded his eight-V100 server to run DeepSeek V4.1 Flash with its most-used experts at NVFP4 precision. The change improves fidelity to Nvidia’s reference build but reduces output speed.
Key points
- The new build stores 5,949 experts, or 38.7% of the total, at NVFP4 precision
- NVFP4 experts receive 80% to 86% of decode routes in one-request tests
- The build adds 1TB of RAM and an RTX 3090
- Two simultaneous requests produce 159 tokens per second in total, versus 118.9 for one request
- Code output is 3% to 30% slower, while legal writing is 9% to 23% slower
Sources
Details
Tom’s Hardware reports that the lawyer, Nick Mermiges of PLI Labs, upgraded his eight-GPU Nvidia V100 server for DeepSeek V4.1 Flash. The system now has 1TB of RAM and an RTX 3090; its V100 GPUs have 32GB each.
The model’s experts remain at 2-bit precision except for 5,949 of the most-used experts, which are stored at NVFP4. That is 38.7% of the experts, while tests found that NVFP4 experts handled 80% to 86% of decode routes with one request at a time. The usage counts included four equally weighted text types, one drawn from Mermiges’s legal harness.
Against Nvidia’s version of the model, the build’s same-next-token rate rose from 92.2% to 95.1%, and KL divergence fell from 0.196 to 0.079 nats. In a 513-question multiple-choice test, the number of answers that differed from the reference fell from 79 to 8; these measures track fidelity, not accuracy, which was about the same across the three builds.
The higher precision comes with slower output: code generation was 3% to 30% slower, and legal writing was 9% to 23% slower, depending on prompt length. Running two requests at once raised total output to 159 tokens per second, compared with 118.9 for one request, but uses more GPU memory and leaves fewer experts at NVFP4.
Written from coverage as of
Images
- Tom's Hardware
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh