Result module

class remayn.result.Result(base_path: str | Path, id: str | None = None, config: dict | None = None)[source]

Represents the result of a experiment. It contains the path where the experiment ResultData is stored, along with the experiment information. The ResultData is only loaded when needed to save memory and time.

base_path

Base path where all the experiments are stored.

Type:

str

id

Unique identifier of the experiment.

Type:

str

config

Dictionary containing the parameters used in the experiment. All the elements in the dictionary must be JSON serializable. Objects contained in this dict should implement a custom __str__ method.

Type:

dict

data\_

Contains the ResultData when loaded or None if it was not loaded yet. This attribute should not be accessed directly. Use get_result() instead to make sure that the ResultData is properly loaded before accessing it.

Type:

Optional[ResultData]

data_md5sum_

md5sum of the ResultData file. It is None if the file was not loaded yet or if creating a new Result that was not saved yet.

Type:

Optional[str]

created_at_

Timestamp when the experiment was created. It is None if the experiment was not saved yet.

Type:

Optional[float]

updated_at_

Timestamp when the experiment was last updated. It is None if the experiment was not saved yet.

Type:

Optional[float]

compare_config(other: Result | dict) → bool[source]

Compare the config of this Result with the config of other Result. It returns True if the configs are equal and False otherwise.

Parameters:

other (Union[Result, dict]) – The other Result object or a dictionary containing the config of the other experiment.

Returns:

True if the configs are equal and False otherwise.

Return type:

bool

copy_to(base_path: str | Path, new_id: str | None = None)[source]

Copies the experiment information and the ResultData to a new directory. It creates a new Result object with the same data and saves it in the new directory. The new Result can have a new unique identifier.

Parameters:

  • base_path (Union[str, Path]) – Base path where the new experiment will be stored. Must be different from the current base path.

  • new_id (Optional[str], optional, default=None) – Unique identifier of the new experiment. If None, the same identifier will be used.

Returns:

A new Result object with the same data saved in the new directory.

Return type:

Result

Examples

>>> from remayn.result import Result
>>> result = Result.load("./results", "123")
>>> new_result = result.copy_to("./new_results")
delete(missing_ok=False)[source]

Deletes the experiment information file (json) and the ResultData file (pickle) from the disk.

Parameters:

missing_ok (bool, optional, default=False) – If True, the method will not raise an error if the files do not exist.

Raises:

FileNotFoundError – If the experiment information file or the ResultData file does not exist and missing_ok is False.

Returns:

True if the files were deleted successfully.

Return type:

bool

get_data(force_reload=False)[source]

Gets the ResultData of the experiment. If it was not loaded yet, it loads it from the disk. If the file was already loaded, it returns the stored object. This method should be used to access the ResultData instead of accessing the data_ attribute directly.

Parameters:

force_reload (bool, optional, default=False) – If True, the ResultData will be reloaded even if it was already loaded. If False, it will only load the ResultData when it has not been loaded yet.

Returns:

The ResultData object containing the results of the experiment. None if the ResultData is empty.

Return type:

ResultData

get_data_path()[source]

Gets the path where the ResultData is stored.

Returns:

Path where the ResultData is stored.

Return type:

Path

get_experiment_info()[source]

Gets all the experiment info as a dictionary, including experiment config, timestamps, md5sum of the ResultData file and the path where the ResultData is stored.

Returns:

Dictionary containing all the experiment information.

Return type:

dict

get_info_path()[source]

Gets the path where the experiment information is stored.

Returns:

Path where the experiment information is stored.

Return type:

Path

get_md5sum()[source]

Gets the md5sum of the ResultData file, which is stored in the experiment information.

Returns:

The md5sum of the ResultData file.

Return type:

str

static load(base_path: str | Path, id: str) → Result[source]

Loads a Result from the disk. It loads the experiment information and the ResultData from the disk and creates a new Result object with the loaded data.

Parameters:

  • base_path (Union[str, Path]) – Base path where all the experiments are stored.

  • id (str) – Unique identifier of the experiment.

Returns:

A new Result object with the loaded data.

Return type:

Result

Raises:

  • FileNotFoundError – If the experiment information file does not exist.

  • ValueError – If the experiment information file is not a valid json file. If the experiment information file does not contain the ‘config’ key. If the experiment information file does not contain the ‘data_md5sum’ key.

Examples

>>> from remayn.result import Result
>>> result = Result.load("./results", "123")
load_data(force=False)[source]

Load the ResultData from the disk. This method reads the ResultData from the disk and stores it in the data_ attribute. It also checks the integrity of the pickle file using the md5sum. This method is called automatically by get_result() when the ResultData is needed. However, you can call it manually to force the loading of the file. If the file was already loaded, this method does nothing, unless force=True is passed as an argument.

Parameters:

force (bool, optional, default=False) – If True, the file will be loaded even if it was already loaded.

Raises:

  • FileNotFoundError – If the ResultData does not exist.

  • ValueError – If the md5sum of the file does not match the one stored in the experiment information.

save()[source]

Saves this Result to the disk. It saves the experiment information in the info_path file and the ResultData in the experiment_info_[‘results_path’] pickle file. If the files already exist, they will be overwritten. If the directory where the files should be saved does not exist, it will be created.

Returns:

The Result object itself.

Return type:

Result

set_data(data: ResultData)[source]

Sets the ResultData of the experiment. This method should be used to set the ResultData instead of setting the data_ attribute directly.

Parameters:

data (ResultData) – The ResultData object containing the results of the experiment.

class remayn.result.ResultData(*, targets: ndarray | list, predictions: ndarray | list, train_targets: ndarray | list | None = None, train_predictions: ndarray | list | None = None, val_targets: ndarray | list | None = None, val_predictions: ndarray | list | None = None, time: float | None = None, train_history: ndarray | list | None = None, val_history: ndarray | list | None = None, best_params: dict | None = None, best_model: object | None = None)[source]

Stores the results of a experiment. ResultData objects only contain data and are usually stored as pickle files.

targets

Numpy array of target values for the test set.

Type:

Union[np.ndarray, list]

predictions

Numpy array of predicted values for the test set.

Type:

Union[np.ndarray, list]

train_targets

Numpy array of target values for the training set.

Type:

Optional[Union[np.ndarray, list]], optional, default=None

train_predictions

Numpy array of predicted values for the training set.

Type:

Optional[Union[np.ndarray, list]], optional, default=None

val_targets

Numpy array of target values for the validation set.

Type:

Optional[Union[np.ndarray, list]], optional, default=None

val_predictions

Numpy array of predicted values for the validation set.

Type:

Optional[Union[np.ndarray, list]], optional, default=None

time

Time taken to run the experiment.

Type:

float, optional, default=None

train_history

Training history of the model, represented as the value of the error on each iteration.

Type:

Optional[Union[np.ndarray, list]], optional, default=None

val_history

Validation history of the model, represented as the value of the error on each iteration.

Type:

Optional[Union[np.ndarray, list]], optional, default=None

best_params

Dictionary of the best parameters found during the experiment. Can be used in case that the experiment employes a cross-validation process.

Type:

Optional[dict], optional, default=None

best_model

Best model found during the experiment.

Type:

Optional[object], optional, default=None

remayn.result.make_result(base_path: str | Path, config: dict, targets: ndarray, predictions: ndarray, train_targets: ndarray | None = None, train_predictions: ndarray | None = None, val_targets: ndarray | None = None, val_predictions: ndarray | None = None, time: float | None = None, train_history: ndarray | None = None, val_history: ndarray | None = None, best_params: dict | None = None, best_model: object | None = None)[source]

Helper function to create a Result object with the given data. It creates a Result object and the associated ResultData. The Result and the ResultData are not saved in the disk. You can call the save() method to save them.

Parameters:

  • base_path (Union[str, Path]) – Path of the main directory that will contain this Result and all the other results related to these experiments.

  • config (dict) – Dictionary containing the parameters used in the experiment.

  • targets (np.ndarray) – Array containing the targets of the experiment. Any shape can be used.

  • predictions (np.ndarray) – Array containing the predictions of the experiment. Any shape can be used.

  • train_targets (Optional[np.ndarray], optional, default=None) – Array containing the training targets of the experiment. Any shape can be used.

  • train_predictions (Optional[np.ndarray], optional, default=None) – Array containing the training predictions of the experiment. Any shape can be used.

  • val_targets (Optional[np.ndarray], optional, default=None) – Array containing the validation targets of the experiment. Any shape can be used.

  • val_predictions (Optional[np.ndarray], optional, default=None) – Array containing the validation predictions of the experiment. Any shape can be used.

  • time (Optional[float], optional, default=None) – Time spent to run the experiment.

  • train_history (Optional[np.ndarray], optional, default=None) – Array containing the training history recorded during the training process. It should be a 1D array with the value of the error on each iteration.

  • val_history (Optional[np.ndarray], optional, default=None) – Array containing the validation history recorded during the training process. It should be a 1D array with the value of the error on each iteration.

  • best_params (Optional[dict], optional, default=None) – Dictionary containing the best parameters found during the experiment. It can be used in case that the experiment employs a cross-validation process. It can be left as None if the experiment does not use a cross-validation process or the cross-validation process is splitted in different experiments.

  • best_model (Optional[object], optional, default=None) – Best model found during the experiment.

Returns:

A new Result object with the given data. The Result is not saved in the disk. You can call the save() method to save it.

Return type:

Result

Examples

>>> import numpy as np
>>> from remayn.result import make_result
>>> targets = np.array([1, 2, 3])
>>> predictions = np.array([1.1, 2.2, 3.3])
>>> config = {"model": "linear_regression"}
>>> result = make_result("results", config, targets, predictions)
>>> result.save()