Lawyer with too many GPUs, part 4: DeepSeek-V4.1-Flash on eight V100s from 2018. 200+ tok/s on code, ~110 on legal writing, 2,200 tok/s prefill at 128K, 524K context, every weight in VRAM
Reported by r/LocalLLaMA1 source ↓
Key points
- The poster says all 248.8 GB of model weights fit in VRAM across eight V100 GPUs
- Reported generation speeds range from 160 to 215 tokens per second for code and 103 to 118 for legal writing
- The setup supports up to 524,288 tokens of context per request
- The poster reports 0.716 balanced accuracy on LegalBench and 89.6% on HumanEval+’s plus tests
- The poster estimates 50–60% of their existing LLM workflows could use this model instead of frontier models
Details
The Reddit poster says the server uses an eight-GPU setup with Tesla V100-SXM2 cards and a quantized DeepSeek-V4.1-Flash model. They report that the model’s 248.8 GB of weights remain in GPU memory during decoding, with one request at a time and a context limit of 524,288 tokens.
In tests reported by the poster, code generation ranged from 215 tokens per second with a 1,000-token prompt to 160 with a 256,000-token prompt. Legal-writing speeds ranged from 118 to 103 tokens per second over those prompt lengths; prefill reached about 2,200 tokens per second at 128,000 and 256,000 tokens. The poster says the tests used greedy decoding, thinking off, and 1,024 generated tokens, with two runs per measurement.
The poster reports a LegalBench balanced accuracy score of 0.716 across 1,292 items, an MMLU-Pro score of 79.9% with a stated margin of ±1.5, and 89.6% on HumanEval+’s plus tests. They estimate that 50–60% of their existing language-model workflows could run on the model, but that estimate is their own assessment.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh