What is Backpropagation?

2020-01-27

Deep learning systems are capable of learning extremely complex patterns, which they achieve by adjusting weights. Deep neural networks, on the other hand, adjust through the process of backpropagation, without which they would not be able to perform tasks such as image recognition and natural language processing. The weights in a deep neural network represent the strength of connections between neural network units. As data passes through the neural network, the weights are calculated and assumptions are made. When the data reaches the final layer of the network, the relationship between these features and the classes in the dataset can be predicted. The difference between the predicted and actual values is the loss/error, and the purpose of backpropagation is to minimize this loss. This is achieved by adjusting the weights of the network to make the assumptions closer to the true relationship between input features. Before backpropagation can be performed on a neural network, regular forward training must be conducted. When creating a neural network, a set of weights is initialized. These weights will change as the network is trained. The forward training process of a neural network can be divided into three steps: neuron activation, neuron propagation, and forward propagation. When training a deep neural network, multiple mathematical functions are utilized. Neurons in a deep neural network consist of input data and activation functions, which determine the values required for activation. The activation value of a neuron is calculated by several components, which are the weighted sum of inputs, and the weights and inputs depend on the index used to calculate the activation node. The backpropagation process determines the final decision of the model training process, which in turn determines the errors in these decisions, calculated by comparing the network's output/decision with the expected output. Once the errors in the network's decisions are calculated, the information is propagated back through the network, and the parameters of the network are also changed. The method used to update the network weights is based on calculus, particularly the chain rule. However, understanding calculus is not a necessary condition for understanding backpropagation. It is sufficient to know that when a neuron provides an output value, the slope of the output value is calculated through the transfer function, resulting in a derived output. When operating on neurons in the output layer, class values are used as the expected values. After calculating the network error, the weights in the network must be updated. "Gradient descent" is the process of updating weights to reduce the error rate. By using the backpropagation method to predict the relationship between neural network parameters and error rate, a gradient descent network is established. Training a network with gradient descent requires calculating weights through forward propagation, propagating errors through backpropagation, and then updating the network weights.

RECENT AI NEWS

RECENT AI TOOLS