Graphical models have become the method of choice for representation of uncertainty in machine learning. Two research issues are currently of major interest in the scientific community: First, much work is devoted to find and analyze more efficient approximate inference algorithms, e.g, loopy belief propagation, variational methods, sampling methods, concave-convex procedure, loop corrections, et cetera. Second, there has been much interest in learning the parameters and the structure of directed graphical models from data. Basically, there are two main paradigms for learning in the machine learning community: generative and discriminative learning. Generative learning is well explored for directed graphical models, whereas, discriminative learning still needs more elaboration. The aim of the proposed research is on discriminative learning of graphical models. In particular, we want to devote significant work on developing discriminative structure and parameter learning algorithms for Bayesian networks and dynamic Bayesian networks. One challenge is certainly the demanding computational complexity. Results of this research are applied to speech and image processing problems, e.g., single channel source separation, multipitch tracking, and multiple object tracking.