Ninfer and GGUF of Qwen3.8-27B-UD-Q3_K_XL ASCII-Condensed
Reported by r/LocalLLaMA1 source ↓
A community member says they published condensed GGUF and Ninfer versions of Qwen3.8-27B-UD-Q3_K_XL, restricted to ASCII and selected Latin characters for use with a 16GB VRAM GPU.
Key points
- The condensed model is available on Hugging Face in GGUF and Ninfer formats
- The creator reports an IFEval score of 84.4%, versus 83.9% for the base model in one run
- The creator reports 75K context and about 60 tokens per second on a 16GB RTX 5060 Ti
Details
The Reddit poster says the model retains ASCII, Latin-1 and Latin Extended-A characters, and was made for use with a 16GB VRAM GPU. The Hugging Face repository includes files and conversion scripts to reproduce the versions; the poster says they also submitted a pull request to the upstream repository.
In one IFEval run, the poster measured 84.4% for the condensed model and 83.9% for the base model, describing the difference as within standard deviation. They also report running a 75K context with rk8v4 KV cache and averaging about 60 tokens per second across 541 IFEval samples on an RTX 5060 Ti with 16GB of VRAM.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh