Dear Reader, I decided to publish an old version of the RProp algorithm I wrote back in 2003 that demonstrates how the algorithm modified to use matrix algebra works. The demo can be downloaded from here, both source files and binaries: https://github.com/bulyaki/TNeuralNetwork Right now I don’t have any means to make changes to it as […]
There is no excerpt because this is a protected post.
Since the RProp algorithm uses if/else conditional statements while determining update values some special helper matrix-functions and helper matrices must be introduced. These functions will allow us to express the conditional statements more elegantly, only using matrix operations. Some matrices needed to make the above functions work: D: decision matrix, M: meta-gradient matrix, U: update […]
In a multi-layered neural network weights and neural connections can be treated as matrices, the neurons of one layer can form the columns, and the neurons of the other layer can form the rows of the matrix. The figure below shows a network and its parameter matrices. The meanings of vectors and matrices above: ninl […]
The RProp algorithm is a supervised learning method for training multi layered neural networks, first published in 1994 by Martin Riedmiller. The idea behind it is that the sizes of the partial derivatives might have dangerous effects on the weight updates. It implements an internal adaptive algorithm which focuses only on the signs of the […]
Preconditioning weights The outcome and speed of a learning process is influenced by the initial state of the network. However, it is impossible to tell which condition will be the most ideal. The commonly accepted way is initializing weights by uniformly distributed random numbers on the (0,1) interval. Preconditioning data Very often the training set […]
I have recently worked on a windows application that consists of multiple services communicating with each other through D-Bus. Each service could run as a console app and during development this is how we were using them. The problem was that managing 7 console windows is not an easy task especially when you have to […]