tanh
/ tanch (or "tan-AYCH") /
Tanh — short for hyperbolic tangent — is sigmoid's close relative. It's the same smooth S-shape, but instead of squashing inputs into the range 0 to 1, it squashes them into the range -1 to 1. Big positive inputs come out near +1, big negative ones near -1, and an input of zero comes out at exactly zero. That one difference — being centered on zero rather than on 0.5 — turns out to matter.
Why does centering help? When a layer's outputs are centered around zero, the next layer receives a balanced mix of positive and negative signals, and the learning algorithm tends to make cleaner, more direct progress. Sigmoid, by contrast, only ever emits positive numbers, which subtly biases the updates and can slow training. For this reason tanh was generally preferred over sigmoid for the hidden layers of older networks, and it still appears inside certain architectures, notably some recurrent networks that process sequences like text or audio.
Tanh shares sigmoid's weakness, though. Its tails are flat too, so for large inputs the slope is near zero and the learning signal can vanish in deep stacks — the same vanishing-gradient trap. That's why for ordinary deep networks today the simpler ReLU usually wins. Think of tanh as the better-behaved member of the S-curve family, but still an S-curve, with the limits that implies.
Feed the numbers -3, 0, and 3 into tanh and you get roughly -0.995, 0, and +0.995. Feed the same numbers into sigmoid and you get roughly 0.05, 0.5, and 0.95. Notice how tanh straddles zero symmetrically while sigmoid sits entirely above it — that symmetry is the practical reason tanh often trained a little better than sigmoid.
Tanh is a zero-centered S-curve (-1 to 1); sigmoid is the same shape shifted up (0 to 1).
Tanh is essentially a rescaled, shifted sigmoid — it's literally 2·sigmoid(2x) − 1. Its zero-centered output is the advantage, but it inherits sigmoid's flat tails and the vanishing-gradient risk that comes with them.