The matrix form of the RProp algorithm

Since the RProp algorithm uses if/else conditional statements while determining update values some special helper matrix-functions and helper matrices must be introduced. These functions will allow us to express the conditional statements more elegantly, only using matrix operations. Some matrices needed to make the above functions work: D: decision matrix, M: meta-gradient matrix, U: update […]

The Resilient Propagation (RProp) algorithm

The RProp algorithm is a supervised learning method for training multi layered neural networks, first published in 1994 by Martin Riedmiller. The idea behind it is that the sizes of the partial derivatives might have dangerous effects on the weight updates. It implements an internal adaptive algorithm which focuses only on the signs of the […]