Obsessive-1.7B: a 1.1 GB web-search subagent
Reported by r/LocalLLaMA1 source ↓
A community developer fine-tuned LFM3-1.7B to answer factual questions using one web search, a verbatim quote and its URL, or report that the search results did not contain the answer.
Key points
- The model is designed to run as a subagent, not as a general chat model
- On 131 fixed-results questions, it found available answers 94% of the time and avoided invented answers
- In live Wikipedia tests, it answered 24 of 120 questions correctly and handled 40 of 40 multi-fact requests
- The developer reports problems with multi-turn chats, English-only use and some missed answers
- Model files and run configurations are available on Hugging Face and in the project repository
Details
The developer says Obsessive-1.7B searches once and returns a short answer, a word-for-word quote and a source URL. If the answer is absent from the results, it is meant to respond, “Not found in the search results.” The model was fine-tuned with LoRA on about 9,000 tool-calling conversations using an RTX 4060 laptop.
In the developer’s fixed-results benchmark of 131 held-out SimpleQA questions, including 29 with the answer removed from search results, Obsessive found answers 94% of the time and correctly reported missing answers 100% of the time; the base model scored 50% and 0%, respectively. The developer also reports zero invented answers for Obsessive in that test, compared with 10% for the base model, and median response times of 2.0 seconds versus 5.1 seconds.
In live Wikipedia searches, Obsessive returned a reply to all 120 questions and answered all 40 multi-fact requests, but its replies contained the correct answer for 24 of the 120 questions. The base model answered 37 correctly, but returned no reply 62% of the time, according to the developer. Obsessive sometimes says an answer is missing when it is present, can quote a neighbouring fact, and may turn reported claims into facts; the developer also says multi-turn chats confuse it.
Written from coverage as of