Models

class gdeep.models.FFNet(arch: Tuple[int, ...] = (2, 3, 3, 2), activation: Callable | None = None)

Custom network class to easily do experiments

It is a simple feed-forward network with a variable number of layers and neurons. The activation function is ReLU by default, but it can be changed by passing a different function as argument. The last layer is not activated.

Args:

arch (Tuple[int, …], optional):

The architecture of the network. Tuple containing the dimension of the layers inside your network. The default is (2, 3, 3, 2). The depth of the network is inferred from the length of the tuple.

activation (Optional[Callable], optional):

The activation function. Defaults to None. If None, ReLU is used. All the layers are activated with the same function. The last layer is not activated.

forward(x: Tensor) → Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class gdeep.models.ModelExtractor(model: Module, loss_fn: Callable[[Tensor, Tensor], Tensor])

This class wraps nn.Modules to extract weights, activations, gradients and decision boundaries

Args:

model:

standard torch module

loss_fn :

loss function

Examples:

from gdeep.analysis import Extractor
# the candidate datum for a question-answering example
x = next(iter(transformed_textts))[0].reshape(1, 2, -1).to(DEVICE)
# the model extractor: you need a trainer and the loss function
ex = ModelExtractor(trainer.model, loss_fn)
# getting the names of the layers
layer_names = ex.get_layers_param().keys()
print("Let's extract the activations of the first attention layer: ", next(iter(layer_names)))
self_attention = ex.get_activations(x)[:2]

get_activations(x: Tensor | List[Tensor]) → List[Tensor]

Compute the activations of self.model with input X

Args:

x :

an example of an input or an input batch of which to compute the activation(s)

Returns:

list:

list of the activation Tensors

get_decision_boundary(input_example: Tensor, n_epochs: int = 100, precision: float = 0.1) → Tensor

Compute the decision boundary for self.model with self.loss_fn

Args:

n_epochs:

number of training cycles to find the decision boundary

input_example:

an example of a single input, to extract the dimensions of the feature space

precision:

parameter to filter the spurious data that are close to te decision boundary, but not close enough

Returns:

Tensor:

the pointcloud defining the decision boundary with dimensions (n_samples, n_features)

get_gradients(batch: Tuple[Tensor | List[Tensor], Tensor], **kwargs) → Tuple[List[Tensor], List[Tensor]]

Returns the averaged gradient of the self.loss_fn. To specify the target variable (e.g. the true class), use the keywords argument target=

Args:

batch:

a data batch on which to compute the average gradients. If the batch has one single item, then it will output the gradients for that single datum

Returns:

list, list:

the gradients for the inputs; the list of tensors, corresponding to the gradient of the weights.

get_layers_grads() → List[Tensor]

Returns the gradients of each layer

Returns:

list:

list of tensors, corresponding to the layer gradients (weights and biases!).

get_layers_param() → Dict[str, Tensor]

Returns parameters of layers

Returns:

dict:

dict of tensors, corresponding to the layer parameters. The key of the dict is the name of the parameters

class gdeep.models.PeriodicNeuralNetwork(nn: Module, boundary_list: List[Tuple[float, float]])

Makes a periodic nn.Module of nn. boundary_list specifies the elementary region the periodic neural network is supported on. This creates neural networks factorized by the orbits of a group effect, which is given by reflection at the edges of the elementary domain. This class can be interpreted as an adapter for periodic neural networks.

Args:

nn :

the standard torch module, your network

boundary_list :

list of pairs of floats, each defining the boundaries of a hypercube

forward(x_cont: Tensor) → Tensor

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class gdeep.models.SaveLayerOutput

class gdeep.models.SaveNodeOutput(entry: int = 0)

Class for saving activations of a node in a neural network.

Args:

entry:

select the entry row in the output of the layer, i.e. the node output

class gdeep.models.SaveOutput

General class for saving outputs of neural networks. Outputs will be stored in a list ‘outputs’