Relu Function

 Preface to Artificial Neural Network

Artificial neural networks are inspired by the natural neurons within the mortal body which spark under certain circumstances performing in a affiliated action performed by the body in response. Artificial neural nets correspond of colorful layers of connected artificial neurons powered by activation functions which help in switching them ON/ OFF. Like traditional machine learning algorithms, then too, there are certain values that neural nets learn in the training phase.

Compactly, each neuron receives a multiplied interpretation of inputs and arbitrary weights which is also added with static bias value ( unique to each neuron subcaste), this is also passed to an applicable activation function which decides the final value to be given out of the neuron. There are colorful activation functions available as per the nature of input values. Once the affair is generated from the final neural net subcaste, loss function ( input vs affair) is calculated and backpropagation is performed where the weights are acclimated to make the loss minimum. Chancing optimal values of weights is what the overall operation is fastening around.

What's activation function?

As mentioned over, activation functions give out the final value given out from a neuron, but what's activation function and why do we need it?
.
So, an activation function is principally just a simple function that transforms its inputs into labors that have a certain range. There are colorful types of activation functions that perform this task in a different manner, For illustration, the sigmoid activation function takes input and maps the performing values in between 0 to 1.

One of the reasons that this function is added into an artificial neural network in order to help the network learn complex patterns in the data. These functions introduce nonlinear real- world parcels to artificial neural networks. Principally, in a simple neural network, x is defined as inputs, w weights, and we pass f (x) that's the value passed to the affair of the network. This will also be the final affair or the input of another subcaste.

Still, the affair signal becomes a simple direct function, If the activation function isn't applied. A neural network without activation function will act as a direct retrogression with limited literacy power. But we also want our neural network to learnnon-linear countries as we give it complex real- world information similar as image, videotape, textbook, and sound.

What's ReLU Activation Function?

ReLU stands for remedied direct activation unit and is considered one of the many mileposts in the deep literacy revolution. It's simple yet really better than its precursor activation functions similar as sigmoid or tanh.

ReLU activation function formula
Now how does ReLU transfigure its input? It uses this simple formula

f (x) = maximum (0, x)
. ReLU function is its outgrowth both are monotonic. The function returns 0 if it receives any negative input, but for any positive value x, it returns that value back. Therefore it gives an affair that has a range from 0 to perpetuity.