RustingStrata vs llama.cpp, DeepSeek-V4.1-Flash Q2 (249 GB) on a single RTX 3060 12 GB with 30 GB RAM
Reported by r/LocalLLaMA1 source ↓
A community post reports that RustingStrata ran a 249 GB DeepSeek-V4.1-Flash model on a system with an RTX 3060 12 GB and 30 GB of RAM, and compares its speed with llama.cpp.
Key points
- RustingStrata ran the model with experts streamed from SSD and a 9 GB VRAM cache
- Reported decode speed was 2–3.6 tokens per second, compared with 0.3–1.5 for llama.cpp
- The test used greedy decoding and had a 4,096-token context cap
- Sampling and tool calls were not yet supported
Details
The r/LocalLLaMA post says RustingStrata loaded the model in IQ2_XXS format, streaming experts from SSD through the page cache and keeping reused experts in a 9 GB VRAM cache. Other operations, including attention and shared matrix-vector calculations, ran on the GPU.
On the same system—an RTX 3060 with 12 GB of VRAM, 30 GB of RAM and 12 CPU cores—the post reports decode speeds of 2–3.6 tokens per second for RustingStrata and 0.3–1.5 for llama.cpp using CPU experts. RustingStrata prefill was about 2 tokens per second for short prompts and about 0.5 at 1,400 tokens; the post says results varied with page-cache state because the model was larger than available RAM.
The post reports top-1 agreement with llama.cpp of 189 out of 192 tokens on short prompts and 21 out of 23 on a 1,425-token prompt. It says the test used greedy decoding; sampling and tool calls were unavailable, and the context limit was 4,096 tokens.
Written from coverage as of