aixd.data

This package contains classes for defining data objects, data blocks, normalizations and transformations that are used to describe datasets.

Dataset

Dataset

This class manages the Dataset. The data, model checkpoints and other logging information resides in the respective folder/file structure:

Data objects

DataObject

Master data object, to define each of the different building blocks that are going to be used to form the design parameters and performance attributes vectors.

DataReal

Real data type.

DataDiscrete

Base class for the discrete type.

DataInt

Integer data type.

DataCategorical

Categorical data type.

DataBool

Boolean type, i.e., categorical type with options 'True', 'False'

Domain definitions

Domain

Abstract base class for domains used by DataObjects.

Options

Defines a set of options, e.g., for categorical variables.

Interval

Defines a closed interval [a, b], for a <= b.

IntervalMasked

A domain that allows to define an interval with several options/masks, not in the interval.

Data blocks

DataBlock

Generic block of data, i.e., a concatenation of different instances of data types

DesignParameters

A data block for design parameters.

PerformanceAttributes

A data block for design parameters.

DesignRepresentation

Additional types, obtained from the design parameters

TransformableDataBlock

A special data block that is transformable.

InputML

A special data block used for the input of the ML model.

OutputML

A special data block used for the output of the ML model.

Custom callbacks

CustomCallback

AnalysisCallback

DataloaderCallback

Callback for pre or post data loading transformation and normalization.

ImportCallback

PostGenerationCallback

Receives the output of the NN, and performs a transformation to it, before returning the value to the user, or before feeding it again to the

SamplingCallback

Transformation

register_transformation

Defines the decorator to register transformations.

resolve_transformation

Simple resolver that returns the transformation by its name.

DataObjectTransform

Abstract base class to implement DataObject transformation.

Log10Transform

Implements a log10 transformation of the data.

SigmoidTransform

Implements a scaled sigmoid transformation of the data.

Standardization

Implements standardization as (x - mean) / std.

MinMaxScaler

Implements a min-max scaling of the data.

ZeroToOne

Implements the zero-to-one (or min-max) normalization as (x - min) / (max - min).

MinusOneToOne

Implements the minus-one-to-one normalization as (x - min) / (max - min).

MaskedMinMaxScaler

Implements min-max scaling of the data with a masked domain.

MaskedZeroToOne

Implements the zero-to-one (or min-max) normalization as (x - min) / (max - min) for DataObject's with a MaskedInterval domain.

MaskedMinusOneToOne

Implements the minus-one-to-one normalization for DataObject's with a MaskedInterval domain.

LabelEncoder

Implements encoding for string to integers.

ToFloat

Implement a transformation to convert integers to floats.

Utils

convert_to

Takes any data format, detect the type, and convert it if neccessary.

combine_formats

Combines data into a single output and converts them into the specified target format.

reformat_dict_to_dataframe

Reformats data formatted as a (list of) dictionaries to a pandas dataframe.

reformat_dataframe_to_dict

Reformats data formatted as a dataframe into a dictionary.

reformat_dataframe_to_dictlist

Reformats data formatted as a dataframe into a dictionary.

reformat_dataframeflat_to_dict

Reformats data stored in a dataframe to a dictionary collated accordingly to the provided list of data objects.

reformat_dataframe_to_dataframeflat

Reformats a dataframe that may represent multidimensional data objects (cells containing lists of values, if dim>1), to a flattened dataframe.

reformat_dataframeflat_to_dataframe

Reformats a flattened dataframe into a dataframe that may represent multidimensional data objects (cells containing n lists of values, if dim>1).

reformat_list_to_array

Reformats data formatted as a nested list into a numpy.ndarray.

reformat_array_to_list

Reformat numpy.ndarray into a nested list.

reformat_list_to_dataframe

Reformats data formatted as a nested list of data into a dataframe.

reformat_dataframe_to_list

Reformats a pandas dataframe into a nested list.

reformat_list_to_dict

Reformats data formatted as a nested list into a dictionary.

reformat_list_to_dictlist

Reformats data formatted as a nested list into a list of dictionaries.

reformat_dictlist_to_dict

The input is a list of dictionaries, where each dictionary, e.g., corresponds to one data sample and the keys correspond to the data object names.

reformat_dict_to_dictlist

In the input dictionary, each key (e.g. corresponding to a data object name) contains a list of items (e.g. corresponding to individual samples).

reformat_array_to_torch

Converts a numpy array to a torch tensor.

reformat_torch_to_array

Converts a torch tensor into a numpy array.