Chairman: Univ.Prof.Dr. L. FICKERT Abstract: In this thesis we investigate centralized and distributed variants of sparse Bayesian learning (SBL), an effective probabilistic regression method used in machine learning. Since inference in an SBL model is not tractable in closed form, approximations are needed. We focus on the variational Bayesian approximation, as opposed to others used in the literature, for three reasons: First, it is a flexible general framework for approximate Bayesian inference that estimates probability densities including point estimates as a special case. Second, it has guaranteed convergence properties. And third, it is a deterministic approximation concept that is even applicable for high dimensional problems where nondeterministic sampling methods may be prohibitive. We resolve some inconsistencies in the literature involved in other SBL approximation techniques with regard to a proper Bayesian treatment and the incorporation of a very desired property, namely scale invariance. More specifically, among all discussed methods, we show that the variational Bayesian approximation is the only approach that can be consistently derived from scaleinvariant model assumptions. A novel fast SBL concept based on fixedpoint analysis of the variational update expressions is presented as a variational counterpart to fast marginal likelihood maximization, a widely known SBL variant. Within this fast framework, which we denote as fast variational SBL (FVSBL), we furthermore show how to incorporate an intuitive tradeoff parameter that allows to balance sparsity versus accuracy. We compare the performance of all methods using real and synthetic data, where we show that FVSBL converges several orders of magnitude faster than ordinary variational SBL. Largescale sensor networks allow to monitor spatial phenomena with unprecedented resolution. Current research focuses on distributed processing performed directly on the sensor nodes with only local communications and no need for data fusion, i.e., the data is not collected and processed at a single point which would be critical to fail. In the SBL context, our objective is to learn a compact model of the true underlying spatially sampled field function based on noisy sensor measurements in a distributed manner. Finding a sparse representation of the data enables its efficient handling and processing, which is a desired property especially for wireless sensor network applications due to energy and bandwidth constraints. We analyze two distributed variants of SBL, which we denote as variational distributed SBL (VdSBL) and fast variational distributed SBL (FVdSBL). Both algorithms are based on consensus methods, which perform distributed averaging based on local message passing between neighboring sensors. VdSBL considers a spatially distributed probabilistic model that is a combination of SBL and consensus propagation. For inference, a variational Bayesian approximation with Gaussian loopy belief propagation as a subroutine is used. The messages involved in this subroutine are directly sent between adjacent sensors and have a communication complexity that is quadratic in the number of model components. In FVdSBL, the FVSBL fixedpoint solution is collaboratively calculated using any type of consensus algorithm, which results in a communication complexity that is only linear in the number of model components for each message. Furthermore, in FVdSBL, we can test and add new components to the model. The computational complexity at each sensor for a single FVdSBL test is quadratic in the number of components, whereas in VdSBL it is cubic for each single message in Gaussian loopy belief propagation. In real and syntheticdata experiments, we compare both distributed algorithms to their centralized counterparts, where similar performance is obtained in most of the cases.

Date with Time
22. April 2013  11:30
Contact
Location
IDEG134, Inffeldgasse 16c, EG
