giagrad.Tensor.kaiming_uniform#

Tensor.kaiming_uniform(neg_slope=0.0, mode='fan_in', nonlinearity='leaky_relu') → Tensor[source]#

Fills Tensor data with the also known He uniform initialization.

Tensor data is filled with values according to the method described in Delving deep into rectifiers using uniform distribution. The resulting tensor will have values sampled from \(\mathcal{U}(-\text{bound}, \text{bound})\) where

\[\text{bound} = \text{gain} \times \sqrt{\frac{3}{\text{fan_mode}}}\]
Parameters:

  • neg_slope (float) – The negative slope of the rectifier used after this layer (only used with ‘leaky_relu’).

  • mode (str, default: 'fan_in') – Either ‘fan_in’ or ‘fan_out’. Choosing ‘fan_in’ preserves the magnitude of the variance of the weights in the forward pass. Choosing ‘fan_out’ preserves the magnitudes in the backwards pass.

  • nonlinearity (str, default: 'leaky_relu') – The non-linear function method name, recommended to use only with ‘relu’ or ‘leaky_relu’.

Examples

>>> Tensor.empty(3, 5).kaiming_uniform(mode='fan_in', nonlinearity='relu')