Layers

Layers#

Here we have our building blocks The Layers

class helixnet.layers.Layer(trainable_params: List[Tensor])#

Parameters:: trainable_params (List[mg.Tensor]) – A list that consists of the parameters that will be trained by the optimiser. Pass an empty list to indicate that it has no trainable parameters

This the base class of all layers that gives them a unique name if they have trainable parameters and if they don’t they will just use their type name without counting.

forward(X: Tensor) → Tensor#

Parameters:: X (mg.Tensor | np.ndarray) – The data that should be forward propagated
Returns:: The prediction of the layer
Return type:: mg.Tensor

this method performs forward propagation

get_config() → dict#: Public method to get the stored configuration.

get_weights() → List[ndarray]#: returns a list of weights of them model.

null_grad()#

This function resets the gradients of parameters. This method should not be modified by children

This function should be used we want to be sure the gradients don’t stack up :return: This method doesn’t return anything

output_shape(prev_shape: Tuple[int] | None = ()) → Tuple[int]#

A simple method that gets the shape of layer’s output

This function should be overloaded in order to get the shape but still can work on it’s own but it’ll need previous shapes in order to work properly

populate_self(population: dict) → None#

This function used in inhereitance in order to to get the arguments in and save self to enable automatic saving

Parameters:: population (dict) – the argument should be local()

predict(*args, **kwargs) → Tensor#

Parameters:: X (mg.Tensor) – The data of forward pass
Returns:: The prediction of the layer
Return type:: mg.Tensor

This function should in inference not in training because it doesn’t track the gradients

set_weights(weights: List[ndarray])#

Sets the weights of them model.

Parameters:: weights (List[np.ndarray]) – This should be the return value of get_weights

total_params()#

This method can calculate the number of parameters in each layer

Return type:: int

Warning

Don’t attempt to modify helixnet.layers.Layer.trainable_params because this might cause the optimizer to lose its state data (e.g., momentum)

Also it’s not recommended to overload helixnet.layers.Layer.predict ,:class:helixnet.layers.Layer.null_grad and helixnet.layers.Layer.total_params to ensure consistent behaviour across the framework

Note

If you can calculate the output of a custom created layer more efficiently you should overload helixnet.layers.Layer.output_shape

class helixnet.layers.Dense(inputs: int, params: int, activation, use_bias: bool = True, dtype=<class 'numpy.float32'>)#

Parameters:

inputs (int) – The size of inputs.
params (int) – The size of parameters. It is the size of the output.
activation – The activation function that will be used with the layer. Any function or object with __call__() method.
use_bias (bool) – Whether to have a bias or not.
dtype – The data type of the parameters of the layers also using data types from MyGrad is preferred over NumPy

A simple dense layer.

forward(X: array)#

Parameters:: X (mg.Tensor) – the inputs of the layer
Return mg.Tensor:: The layer predictions

output_shape(prev_shape: Tuple[int] | None = None) → Tuple[int]#

A simple method that gets the shape of layer’s output

This function should be overloaded in order to get the shape but still can work on it’s own but it’ll need previous shapes in order to work properly

class helixnet.layers.Conv2D(input_channels: int, output_channels: int, kernel_size, stride=1, padding=0, activation=None, use_bias: bool = True)#

Parameters:

input_channel (int) – the expected number of input channels.
output_channel (int) – The number of channels should the layer output.
kernel_size (int) – the size of the kernel.
activation – The activation function that will be used with the layer. Any function or object with __call__() method.
use_bias (bool) – Whether to have a bias or not.

helixnet.layers.Conv2D assumes input data is of: shape (N, C_in, H, W):

N: batch size

C_in: number of input channels

H: height of the input feature map

W: width of the input feature map

forward(X: Tensor) → Tensor#

Performs a forward pass.

Also the images needs t be in the following shape (batch_size, color_channels, length, width)

class helixnet.layers.MaxPooling2D(pool_size, stride=None)#

Parameters:: pool_size (int | Tuple[int, int]) – the pool size can be integer for square pools and can be a tuple for rectangular tuples

A layer to perform max pooling over a 4D input (No_samples, Channels, Height, Width).

forward(X: Tensor) → Tensor#: Applies the max pooling operation.

class helixnet.layers.Flatten#

A simple flatten layer that turns it’s inputs into a flat layer

forward(X: Tensor) → Tensor#

Parameters:: X (mg.Tensor) – The tensor that will be flattened
Return mg.Tensor:: A flat tensor

Takes an input of shape (N, C, H, W) and flattens it to a shape of (N, C*H*W).

output_shape(prev_shape: Tuple[int] = ()) → Tuple[int]#

A simple method that gets the shape of layer’s output

This function should be overloaded in order to get the shape but still can work on it’s own but it’ll need previous shapes in order to work properly

class helixnet.layers.Reshape(target_shape)#

Reshapes the input tensor to the specified shape.

Parameters:: target_shape (tuple[int]) – The shape you want the data to be converted to.

forward(X: Tensor) → Tensor#

Parameters:: X (mg.Tensor | np.ndarray) – The data that should be forward propagated
Returns:: The prediction of the layer
Return type:: mg.Tensor

this method performs forward propagation

output_shape(prev_shape: Tuple[int] | None = ()) → Tuple[int]#

A simple method that gets the shape of layer’s output

This function should be overloaded in order to get the shape but still can work on it’s own but it’ll need previous shapes in order to work properly

class helixnet.layers.Embedding(vocab_size, dim)#

Word embedding layer

Parameters:

vocab_size (int) – The size of vocabulary
dim (int) – the number of output dimensions

forward(X: Tensor)#

Parameters:: X (mg.Tensor | np.ndarray) – The data that should be forward propagated
Returns:: The prediction of the layer
Return type:: mg.Tensor

this method performs forward propagation

class helixnet.layers.InputShape(shape: Tuple[int], ensure_shape: bool | None = True)#

A very simple layer designed just for model designing where you might need in order to determine the model shapes and it can make sure the input shape is correct where it the shape of data is (X, D1, D2, D3, … ) where it ignores the the first dimension because it is the number of samples.

forward(X: Tensor) → Tensor#: Just returns the inputs

output_shape(prev_shape: Tuple[int] | None = None) → Tuple[int]#: returns the input shape of layers

class helixnet.layers.LSTMLayer(input_size: int, hidden_size: int, return_sequences: bool = True)#

An LSTM layer that processes a sequence of inputs by unrolling an LSTMCell over time.

forward(X: Tensor) → Tensor#

Processes a sequence of inputs.

Parameters:: X (mg.Tensor) – The input sequence, shape (N, seq_len, input_size).
Return mg.Tensor:: The sequence of hidden states, shape (N, seq_len, hidden_size), or the final hidden state, shape (N, hidden_size).

class helixnet.layers.LSTMCell(input_size: int, hidden_size: int)#

A single cell of an LSTM. Performs the computation for one timestep.

forward(x_t: Tensor, h_prev: Tensor, C_prev: Tensor)#

Performs a forward pass for a single timestep.

Parameters:

(mg.Tensor) (C_prev) – Input for the current timestep, shape (N, input_size).
(mg.Tensor) – Hidden state from the previous timestep, shape (N, hidden_size).
(mg.Tensor) – Cell state from the previous timestep, shape (N, hidden_size).

Return Tuple[mg.Tensor, mg.Tensor]:

The new hidden state (h_next) and cell state (C_next).

class helixnet.layers.BatchNorm(input_shape: Tuple[int], momentum=0.99, epsilon=1e-07)#

Performs batch normalization.

Parameters:

input_shape (Tuple[int]) – The shape of the input of the data
momentum (float) – The momentum of the layer
epsilon (float) – A simple number for numerical stability

forward(X) → Tensor#

Parameters:: X (mg.Tensor | np.ndarray) – The data that should be forward propagated
Returns:: The prediction of the layer
Return type:: mg.Tensor

this method performs forward propagation

predict(X, *args, **kwargs) → Tensor#

Normalize the data without updating the running mean and variance

Parameters:: X (mg.Tensor) – The input of the data

class helixnet.layers.Dropout(proba: float)#

A dropout layer

Parameters:: proba (float) – The percentage of inactive neurons

forward(X)#

Parameters:: X (mg.Tensor | np.ndarray) – The data that should be forward propagated
Returns:: The prediction of the layer
Return type:: mg.Tensor

this method performs forward propagation

predict(X)#: In the method helixnet.layers.Dropout.predict the helixnet.layers.Dropout won’t perform the dropout and pass the inputs directly.

class helixnet.layers.DenseTranspose(layer: Dense, activation=None, use_bias=None)#

This layer can be used to tie the weights only of a dense layer useful in autoencoders

This layer doesn’t tie the bias and can create it’s own or not

Parameters:

layer (Dense) – The layer that its weights will be tied.
activation – The activation function that will be used by the layer. But if it’s none it will use the tied layer’s activation function.
use_bias (bool) – If you want the layer to created it’s own bias.

forward(X) → Tensor#

Parameters:: X (mg.Tensor | np.ndarray) – The data that should be forward propagated
Returns:: The prediction of the layer
Return type:: mg.Tensor

this method performs forward propagation

output_shape(prev_shape: Tuple[int] | None = ()) → Tuple[int]#

A simple method that gets the shape of layer’s output

This function should be overloaded in order to get the shape but still can work on it’s own but it’ll need previous shapes in order to work properly

class helixnet.layers.ConvTranspose2D(input_channels: int, output_channels: int, kernel_size, stride=1, activation=None, use_bias: bool = True)#

Performs a 2D transpose convolution (deconvolution), used for upsampling. This implementation works by first upsampling the input with zero-insertion, and then performing a standard convolution.

forward(X: Tensor) → Tensor#: Performs the transpose convolution using a two-step process.

Layers

Contents

Layers#