giagrad.nn.Linear#

class giagrad.nn.Linear(*args, **kwargs)[source]#

Densely-connected Neural Network layer: \(y = xA^T + b\).

Both w and b are initialized from \(\mathcal{U}(\sqrt{-k}, \sqrt{k})\), where \(k = \frac{1}{\text{in_features}}\).

Inherits from: Module.

Variables:

  • w (Tensor) – Learnable weights of the layer of shape \((\text{out_features}, \text{in_features})\).

  • b (Tensor, optional) – Learnable bias of the layer. Only exists when bias is True.

Parameters:

  • in_features (int) – Size of each input sample.

  • out_features (int, optinal) – Size of each output sample.

  • bias (bool, default: True) – If set to False, the layer will not learn an additive bias.

Shape:

  • Input – \((*, H_{in})\) where \(*\) means any number of dimensions including none and \(H_{in} = \text{in_features}\).

  • Output – \((*, H_{out})\) where all but the last dimension are the same shape as the input and \(H_{out} = \text{out_features}\).

Examples

>>> layer = nn.Linear(10, 5)
>>> x = Tensor.empty(2, 10).uniform()
>>> y = layer(x)
>>> y.shape
(2, 5)

Tensors can also be initialized lazily, passing only one value x is equivalent to out_features=x.

>>> layer = nn.Linear(5)
>>> x = Tensor.empty(2, 10).uniform()
>>> y = layer(x)
>>> y.shape
(2, 5)