giagrad.Tensor.mish#

Tensor.mish(beta=1.0, limit=20.0) → Tensor[source]#

Returns a new Tensor with element-wise Mish function. See Mish.

\[out_i = data_i \times \text{tanh} \left( \text{softplus}(data_i) \right)\]

Parameters:

beta¶ (float) – The \(\beta\) value for the Softplus formulation.
limit¶ (float) – Data times beta above limit reverts to a linear function in Softplus formulation.

See also

softplus()

Examples

>>> t = Tensor.empty(2, 3).uniform(-5, 5)
>>> t
tensor: [[-1.3851491  1.2130666 -4.9049625]
         [ 2.6859815 -4.845946   2.1385565]]
>>> t.mish()
tensor: [[-0.3043592   1.0920179  -0.03620962]
         [ 2.6642     -0.03794033  2.0915585 ]] fn: Mish(beta=1.0, lim=20.0)