A 0.1B model just killed 8B embedding models... I tried it out.
, updated First seen Updated Limited coverage, tapering
Models
Reported by r/LocalLLaMA1 source ↓
Key points
- The developer says SPLADE beat Qwen3-Embedding-0.6B on a few hundred queries
- The SPLADE model used less than 1.5 GB of VRAM during inference, the developer reports
- The developer says SPLADE ran on an M5 MacBook Air without out-of-memory errors
- The developer says combining SPLADE with dense embeddings performed worse than SPLADE alone
Details
The r/LocalLLaMA poster says the test corpus consisted of documents on their Mac, mostly in Korean with some English. They compared KURE-sparse-nano, a SPLADE model, with Qwen3-Embedding-0.6B and report that SPLADE had better retrieval results and similar query speed.
The developer says the dense model was difficult to run on an M5 MacBook Air with 16 GB of memory and that the computer rebooted due to heat. KURE-sparse-nano completed the test without out-of-memory errors or warnings and used less than 1.5 GB of VRAM, according to the post.
The developer says they open-sourced an implementation that includes optimized inference and indexing code.
Written from coverage as of