A developer on r/LocalLLaMA described a sandbox simulation with 50–100 agents running on OpenClaw. To limit LLM calls, the developer uses a decision model to determine whether an agent needs to respond to a signal.
The developer said the current model, Jev, takes about 0.5–2 seconds to respond over the network. They asked whether a smaller locally hosted model, including Needle 3, could be faster and what GPU resources it would require, or whether it could run on a CPU. The post did not report any tests or results comparing local and API latency.