https://www.nbshare.io/notebook/751082217/Activation-Functions-In-Python/

University of Minnesota - Twin Cities

Luckily it can be categorized into 2 types:

- Linear Activation Function

- Non-linear Activation Function

Types of Activation Function

- Sigmoid or Logistic Activation Function
- Tanh Activation Function
- ReLU (Rectified Linear Unit) Activation Function
- Softmax Activation Function

Non-Linear Activation Functions

The identity function is the only function belonging to the linear activation-function type. It is defined as $$f(x) = x$$, so the output equals the input unchanged. Unlike non-linear activation functions such as sigmoid or tanh, its output is not compressed into a bounded range.

Learn Before

Related