giagrad.Tensor.quick_gelu#

Tensor.quick_gelu() → Tensor[source]#

Returns a new Tensor with element-wise Quick GELU. See GELU.

Quick GELU is an approximation of GELU through silu() with alpha = 1.702 to ease GELU’s computational complexity.

Examples

>>> t = Tensor.empty(2, 3).uniform(-1, 1)
>>> t
tensor: [[ 0.62271285  0.37412217 -0.6465454 ]
         [-0.9013401  -0.02915052 -0.9814293 ]]
>>> t.quick_gelu()
tensor: [[ 0.4624659   0.2446833  -0.16141725]
         [-0.15989538 -0.01421376 -0.15543076]] fn: QuickGELU