CrowdGPT v2.6 - The collaborative LLM
Reported by r/LocalLLaMA1 source ↓
A CrowdGPT developer announced v2.6, an update to the open collaborative training project. The developer says contributors trained the model on 4.5 billion new tokens in six days.
Key points
- CrowdGPT aims to collaboratively train a 1-billion-parameter model using contributors’ PCs
- The developer reports 4.5 billion new tokens were used for training in six days
- Three clients now cross-check training on the same data to make malicious contributions harder
- The project has a data-submission pipeline; 400 KB had been submitted at the time of the post
- The developer also released CrowdBench, a collaborative benchmark with deterministic-answer questions
Details
The v2.6 update adds server-side contribution checks, client changes including GPU selection and installation improvements, and fixes to server and client code. The developer says the changes followed security feedback and caused some earlier training contributions to become stale.
The developer says three clients now train on the same data for Byzantine cross-checks. They estimate that a malicious client could contribute only 2–3% malicious data per round without being banned, compared with 100% before; they caution that the system is not a fully secure federated-learning framework. The update also adds signed client releases and a pipeline for submitted data, which the developer says will be checked and curated before training.
Written from coverage as of