Delta Inference Engine v0.93 adds Unsloth quant support and lower-RAM operation
Reported by r/LocalLLaMA1 source ↓
The community developer behind Delta Inference Engine says version 0.93 supports Unsloth quantized models and configurations with 64 GB of RAM. In the developer’s tests, one RTX 3090 ran an Unsloth IQ2 model at 41 tokens per second with 192 GB of RAM.
Key points
- The developer reports 57 correct answers on a 99-question test, versus 52 for the prior quant and 59 for the full model
- The developer says a short prompt’s time to first token fell from about 3.8 seconds to 2.3
- The listed minimums are Windows 10 or 11, an Nvidia RTX GPU with at least 12 GB VRAM, and 64 GB RAM
- A two-GPU setup using an RTX 3060 and RTX 3090 delivered a reported 7% speed increase
Details
The release supports 64, 96, 128 and 192 GB RAM configurations.
On the developer’s 99-question benchmark, the new UD-IQ2_XXS quant scored 57 correct, compared with 52 for the previous ds4 Q2 quant and 59 for the full model.
The developer also reports that reading a roughly 180-token prompt before the first response fell from about 3.8 seconds to 2.3 seconds. The project is available under an MIT license; the listed requirements include Windows 10 or 11, an Nvidia RTX GPU with a driver version of at least 580, and a fast NVMe drive. AMD GPUs are not supported, according to the post.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh