giagrad.nn.QuickGELU

class giagrad.nn.QuickGELU(*args, **kwargs)

Applies \(\text{GELU}(x) = x \times \Phi(x)\) activation function with SiLU approximation.

See also

giagrad.Tensor.quick_gelu()