Abstract

Gaussian mixture models are used in many applications to capture the underlying distribution of the given data. Gaussian mixture models are usually learned according to the generative paradigm, for instance in a maximum likelihood sense, such that the available data is described well by the model. Nevertheless, when generatively learned models are used to perform classi- fication, the resulting classification errors are in practice typically higher than the classification errors of discriminatively trained models. Recent work presented several ways to learn Gaussian mixture models discriminatively to solve this problem. However, none of these approaches takes the likelihood into account. Consequently, the likelihood is typically small and the probabilistic interpretation of the model suffers. A good probabilistic interpretation of the model enables classification of data with missing features in a mathematical sound way whereas discrimina- tive models often rely on heuristics such as imputation techniques. In this thesis we present a hybrid generative-discriminative approach to learn Gaussian mixture models. This is achieved by optimizing an objective that trades off between a generative likelihood term and a discrimi- native margin term. We show the classification performance on synthetic and real world data. We demonstrate the capabilities of hybrid Gaussian mixture models when classifying data with missing features and show how unlabeled data can be used to improve the accuracy of the clas- sifier, i.e. semi-supervised learning. The resulting models improve the performance of purely generatively learned Gaussian mixture models and achieve a high accuracy in the presence of missing data. The hybrid model is compared to support vector machines and other state of the art Gaussian mixture model classifiers.