Learning of Bayesian Network Classi ers and Sum- Product Networks

Discriminative learning of Bayesian networks (BNs) for classi fication tasks is often bene ficial
compared to generative learning. This is particularly true in case of model mismatch, i.e. when
the BN cannot represent the true data distribution. In the past, we developed maximum margin
parameter learning for Bayesian network classifi ers and Gaussian Mixture models. Furthermore, we
used the margin objective for approximate and exact structure learning. This research is extended
within this proposal. The focus is three-fold: (i) Extension of margin-based parameter learning to
a hybrid paradigm merging the advantages of generative and discriminative learning. We aim at
extending our learning framework to semi-supervised, missing features, and latent variable scenarios.
This requires efficient inference during iterative parameter optimization. Additionally, both
the discriminative and hybrid learning approach are introduced to potentially deep sum-product
networks (SPNs). They explicitly represent the inference process, i.e. structures (including latent
variables) exhibiting computational benefi ts for inference can be exploited. (ii) Discriminative
search-and-score structure learning in BNs is time-consuming. We are interested in approximating
the non-decomposable discriminative score by a decomposable surrogate to ease the computational
costs for score evaluation in BNs. Furthermore, we aim at developing structure learning algorithms
for SPNs introducing a global scoring function with an inference cost penalty. (iii) To consolidate
SPNs with respect to empirical performance we will compare all developed models to popular generative
and discriminative models from the deep community, i.e. restricted Boltzmann machine,
auto-encoders, deep belief networks, multi-layer perceptron. Additionally, one particularly interesting
recent deep model generative stochastic networks is considered.

Funding Program: 
FWF (Austrian Science Fund)
Research Area: 
2015 - 2018