aixd.data

This package contains classes for defining data objects, data blocks, normalizations and transformations that are used to describe datasets.

Dataset

Dataset

This class manages the Dataset. The data, model checkpoints and other logging information resides in the respective folder/file structure:

Data objects

`DataObject`	Master data object, to define each of the different building blocks that are going to be used to form the design parameters and performance attributes vectors.
`DataReal`	Real data type.
`DataDiscrete`	Base class for the discrete type.
`DataInt`	Integer data type.
`DataCategorical`	Categorical data type.
`DataBool`	Boolean type, i.e., categorical type with options 'True', 'False'

Domain definitions

`Domain`	Abstract base class for domains used by DataObjects.
`Options`	Defines a set of options, e.g., for categorical variables.
`Interval`	Defines a closed interval [a, b], for a <= b.
`IntervalMasked`	A domain that allows to define an interval with several options/masks, not in the interval.

Data blocks

`DataBlock`	Generic block of data, i.e., a concatenation of different instances of data types
`DesignParameters`	A data block for design parameters.
`PerformanceAttributes`	A data block for design parameters.
`DesignRepresentation`	Additional types, obtained from the design parameters
`TransformableDataBlock`	A special data block that is transformable.
`InputML`	A special data block used for the input of the ML model.
`OutputML`	A special data block used for the output of the ML model.

Custom callbacks

`CustomCallback`
`AnalysisCallback`
`DataloaderCallback`	Callback for pre or post data loading transformation and normalization.
`ImportCallback`
`PostGenerationCallback`	Receives the output of the NN, and performs a transformation to it, before returning the value to the user, or before feeding it again to the
`SamplingCallback`

Transformation

`register_transformation`	Defines the decorator to register transformations.
`resolve_transformation`	Simple resolver that returns the transformation by its name.
`DataObjectTransform`	Abstract base class to implement DataObject transformation.
`Log10Transform`	Implements a log10 transformation of the data.
`SigmoidTransform`	Implements a scaled sigmoid transformation of the data.
`Standardization`	Implements standardization as (x - mean) / std.
`MinMaxScaler`	Implements a min-max scaling of the data.
`ZeroToOne`	Implements the zero-to-one (or min-max) normalization as (x - min) / (max - min).
`MinusOneToOne`	Implements the minus-one-to-one normalization as (x - min) / (max - min).
`MaskedMinMaxScaler`	Implements min-max scaling of the data with a masked domain.
`MaskedZeroToOne`	Implements the zero-to-one (or min-max) normalization as (x - min) / (max - min) for DataObject's with a MaskedInterval domain.
`MaskedMinusOneToOne`	Implements the minus-one-to-one normalization for DataObject's with a MaskedInterval domain.
`LabelEncoder`	Implements encoding for string to integers.
`ToFloat`	Implement a transformation to convert integers to floats.

Utils

`convert_to`	Takes any data format, detect the type, and convert it if neccessary.
`combine_formats`	Combines data into a single output and converts them into the specified target format.
`reformat_dict_to_dataframe`	Reformats data formatted as a (list of) dictionaries to a pandas dataframe.
`reformat_dataframe_to_dict`	Reformats data formatted as a dataframe into a dictionary.
`reformat_dataframe_to_dictlist`	Reformats data formatted as a dataframe into a dictionary.
`reformat_dataframeflat_to_dict`	Reformats data stored in a dataframe to a dictionary collated accordingly to the provided list of data objects.
`reformat_dataframe_to_dataframeflat`	Reformats a dataframe that may represent multidimensional data objects (cells containing lists of values, if dim>1), to a flattened dataframe.
`reformat_dataframeflat_to_dataframe`	Reformats a flattened dataframe into a dataframe that may represent multidimensional data objects (cells containing n lists of values, if dim>1).
`reformat_list_to_array`	Reformats data formatted as a nested list into a numpy.ndarray.
`reformat_array_to_list`	Reformat numpy.ndarray into a nested list.
`reformat_list_to_dataframe`	Reformats data formatted as a nested list of data into a dataframe.
`reformat_dataframe_to_list`	Reformats a pandas dataframe into a nested list.
`reformat_list_to_dict`	Reformats data formatted as a nested list into a dictionary.
`reformat_list_to_dictlist`	Reformats data formatted as a nested list into a list of dictionaries.
`reformat_dictlist_to_dict`	The input is a list of dictionaries, where each dictionary, e.g., corresponds to one data sample and the keys correspond to the data object names.
`reformat_dict_to_dictlist`	In the input dictionary, each key (e.g. corresponding to a data object name) contains a list of items (e.g. corresponding to individual samples).
`reformat_array_to_torch`	Converts a numpy array to a torch tensor.
`reformat_torch_to_array`	Converts a torch tensor into a numpy array.