Variational Bayesian Reservoir Computing and its Applications to Handwriting Recognition

Project Type: Master/Diploma Thesis
Student: Zechner Christoph
Mentor: Gernot Kubin


 Reservoir computing principles - and in particular - Echo State Networks (ESNs) have been shown to work well for many different applications, such as chaotic time series prediction, speech recognition or adaptive control. However, in some cases standard ESNs show only poor learning performance. For this reason, several model extensions to classical ESN learning have been proposed in the literature, such as filter neurons or tunable delay & sum readouts, to name just a few. Furthermore, ESN learning algorithms require appropriate regularization techniques, especially if large reservoirs are used. Clearly, ESN learning algorithms, which can also deal with tunable model extensions as well as an automatic regularization would be of general interest. However, jointly solving both requirements turns out to be analytically complex and thus, this work utilizes the variational Bayesian (VB) inference framework, to obtain approximate learning algorithms for three different ESN models. Using real-world handwriting trajectories, this thesis tries to find out, if and how the VB algorithms can improve ESN learning performance.