Tanh Activation Function

With a difference in output range of -1 to 1, the tanh activation function is remarkably similar to the sigmoid/logistic activation function, and even has the same S-shape. The larger the input (more positive), the closer the output to 1.0, and the smaller the input (more negative), the closer the output to -1.0.

Mathematically it can be represented as:

The following are some of the benefits of using this activation function:

We can simply map the output values as strongly negative, neutral, or very positive because the tanh activation function's output is Zero centred.

Because its values range from -1 to, it's commonly utilised in hidden layers of neural networks. As a result, the hidden layer's mean is 0 or extremely close to it. It aids in data centering and makes learning the next layer much simpler.

To understand the tanh activation function's limits, look at its gradient.

As you can see, it, like the sigmoid activation function, has the problem of vanishing gradients. In addition, the tanh function has a substantially steeper gradient than the sigmoid function.