The designed algorithm of deep neural network is a generalization of the back-propagation error (BPA) learning algorithm to a larger number of hidden layers of the neural network (network depth), complemented by a forward pre-learning (depth learning) neural network algorithm based on layered autocoders on top of each other. The depth of the network is in the order of tens of layers or more. A back-propagation algorithm is used to network training. The training is performed in two phases, i.e., first pretraining the network in the forward direction, e.g., using autoencoders (learning without a tutor), and then training the network in the backward direction (learning with a tutor), thus eliminating the damping of backpropagation error. The network will be used to estimate the reliability of power equipment.
The test of the designed algorithm was performed on a deep neural network with 10 layers, each with 10 neurons, except for the output layer with only 3 neurons. The aim of the network was to classify objects with 10 attributes to 3 categories.
The algorithm itself ran as previously described. The network was initially adapted by a generalization of the back-propagation error (BPA) learning algorithm. The following stages were deep learning: pre-training in the forward direction and training in the backward direction. The pre-training was done sequentially for layers 1 to 4, 1 to 5, etc., up to 1 to 9; using the auto-associative function with two layers of weights. The first layer was the corresponding layer of the network and the second layer was a fictive layer, artificially added in order the complete the sequentially arranged autoencoders. The following training was done in the same way.
Results show that the ideal technique to minimize the amount of pre-trained layers is to pre-train the layers 1 to 5, as the weights up to the last, 9th layer, do change enough.
Downloads:
- User guide (in Czech)
- Licence agreement and trial version
The software has been developed with a financial support of the Technology Agency of the Czech Republic under the research project No. TK04020003.
