Tiiny AI Performance tests. I just got my Tiiny, and here's what it can do
Reported by r/LocalLLaMA1 source ↓
A Tiiny Pocket owner reported speed and coding tests of three local models on the device. The 35B model generated 63.6 tokens per second, while the 120B model passed all 10 coding tests but averaged 16 tokens per second.
Key points
- The 35B model generated 63.6 tokens per second; the 9B model reached 75
- The 120B model averaged 15.97 tokens per second and took 33 seconds for a benchmark request
- In a four-file coding task, the 120B model passed 10 of 10 tests; the 35B passed 9
- The owner says TiinyOS did not support importing custom or Hugging Face models at the time of testing
Sources
Details
The Reddit poster said the tests used a Tiiny Pocket running TiinyOS 0.1.36. The device has 32GB of SoC memory and 48GB of dedicated NPU memory, according to the post; the owner described it as about 5.6 by 3.1 by 0.9 inches and 300 grams.
For speed tests, the owner used prompts of about 4,000 input tokens, disabled prompt caching and reported the median of five runs. The 9B model processed output at 75 tokens per second, the 35B model at 63.6, and the 120B model at 15.97; the 120B benchmark took 33.07 seconds and produced 390 output tokens.
In a separate coding test, the models were asked to make a four-file Python inventory and order management package. The owner tested the generated code on a Mac with 10 automated tests: the 120B model passed all 10 in about 229 seconds, while the 35B model passed nine in about 235 seconds. The 9B model did not produce a single gradable result across two attempts.
The poster said the 120B model took about 75 seconds to load in that run and used almost all of the dedicated NPU memory. The post also said users could not yet import their own models or install models from Hugging Face, and that the Tiiny team had mentioned a model import tool was planned.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh