A developer releases inside-llm, a browser tool for inspecting and comparing language models
Reported by r/LocalLLaMA1 source ↓
The open-source inside-llm tool runs GGUF models through llama.cpp and visualizes token probabilities, sampling steps and differences between two models as they generate text.
Key points
- It can compare two models token by token in lockstep or let each generate independently
- Both models must share a tokenizer and fit in VRAM at the same time
- The tool runs locally on Linux and Windows, with GPU or CPU execution
- It is text-only and does not support split GGUF files or embedding models
Details
The developer says inside-llm displays generated text and token probabilities, marks positions where two models would choose different tokens, and shows candidate distributions and divergence metrics. Users can inspect sampling stages, branch from candidate tokens and view layer predictions through an optional logit lens.
The tool uses a C++17 process linked to llama.cpp and a browser interface. It can record sessions for later replay without a model or GPU, and is released under the MIT license. The developer says it listens only on the local machine and has no accounts or telemetry.
The project offers packages for Linux with glibc 2.35 or later and Windows, with CUDA or Vulkan support. If no dedicated GPU is available, it can run on the CPU, which the developer says is slow.
Written from coverage as of