A paper introduces e-means, a mini-batch k-means method using exponential moving averages. It reports matching or beating Lloyd’s 25-epoch loss in 1.4 to 2.6 epochs across embedding datasets.
The paper describes e-means as a mini-batch k-means algorithm that updates cluster centers using exponential moving averages. Its authors report that it reaches lower loss than Lloyd’s k-means, and often beats k-means++, across a range of embedding datasets.
The authors report that e-means matches or beats the loss of Lloyd’s method after 25 epochs in 1.4 to 2.6 epochs, corresponding to 10–45× GPU wall-clock speedups against the implementations they tested. They also describe scaling laws for choosing the method’s decay rate and batch size without a hyperparameter sweep.