giagrad.nn.SoftPlus#

class giagrad.nn.SoftPlus(*args, **kwargs)[source]#

Applies element-wise \(\text{SoftPlus}(x) = \frac{1}{\text{beta}} \cdot \log(1 + \exp(\text{beta} \times data_i))\).

For numerical stability the implementation reverts to the linear function when \(data_i \times \text{beta} > \text{limit}\).

See also

giagrad.Tensor.softplus()

Variables:

  • beta (float) – The \(\beta\) value for the Softplus formulation.

  • limit (float) – Data times beta above this reverts to a linear function.