Tanh Function

The tanh activation function is just another possible function that can be used as anon-linear activation function between layers of a neural network. It shares a many effects in common with the sigmoid activation function. Unlike a sigmoid function that will collude input values between 0 and 1, the Tanh will collude values between-1 and 1. Analogous to the sigmoid function, one of the intriguing parcels of the tanh function is that the outgrowth of tanh can be expressed in terms of the function itself.

When to use which Activation Function in a Neural Network?

Specifically, it depends on the problem type and the value range of the anticipated affair. For illustration, to prognosticate values that are larger than 1, tanh or sigmoid aren't suitable to be used in the affair subcaste, rather, ReLU can be used. On the other hand, if the affair values have to be in the range () or (-1, 1) also ReLU isn't a good choice, and sigmoid or tanh can be used then. While performing a bracket task and using the neural network to prognosticate a probability distribution over the mutually exclusive class markers, the softmax activation function should be used in the last subcaste. Still, regarding the retired layers, as a rule of thumb, use ReLU as an activation for these layers.

In the case of a double classifier, the Sigmoid activation function should be used. The sigmoid activation function and the tanh activation function work terribly for the retired subcaste. For retired layers, ReLU or its better interpretation dense ReLU should be used. For a multiclass classifier, Softmax is the best- used activation function. Though there are further activation functions known, these are known to be the most habituated activation functions.

Tanh Activation Function

import matplotlib.pyplot as plt
import numpy as np
def tanh(x):
    t=(np.exp(x)-np.exp(-x))/(np.exp(x)+np.exp(-x))
    dt=1-t**2
    return t,dt
z=np.arange(-4,4,0.01)
tanh(z)[0].size,tanh(z)[1].size
fig, ax = plt.subplots(figsize=(9, 5))
ax.spines['left'].set_position('center')
ax.spines['bottom'].set_position('center')
ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')
ax.xaxis.set_ticks_position('bottom')
ax.yaxis.set_ticks_position('left')
ax.plot(z,tanh(z)[0], color="#307EC7", linewidth=3, label="tanh")
ax.plot(z,tanh(z)[1], color="#9621E2", linewidth=3, label="derivative")
ax.legend(loc="upper right", frameon=False)
fig.show()