Parameter Estimation in Gaussian Mixture Models with Malicious Noise, without Balanced Mixing Coefficients

Abstract

We consider the problem of estimating the means of components in a noisy 2-Gaussian Mixture Model (2-GMM) without balanced weights, where the noise is of an arbitrary distribution. We present a robust algorithm to estimate the parameters, together with upper bounds on the numbers of samples required for the good estimates, where the bounds are parametrised by the dimension, ratio of the mixing coefficients, the separation of the two Gaussians in terms of Mahalanobis distance, and a condition number of the covariance matrix. In theory, this is the first sample-complexity result for Gaussian mixtures corrupted by adversarial noise. In practice, our algorithm outperforms the vanilla Expectation-Maximisation (EM) algorithm by orders of magnitude in terms of estimation error.