For the tasks of preprocessing pandas data and speeding up experiments, we have the Preprocessor class and a number of base classes with single functionality at preprocessing.
These methods should be implemented for spark dataframes, in the same paradigm as we have for the Designer and the Splitter.
At this moment, the implementation of the following methods is essential:
- Aggregation
- Outliers removal (robust)
- CUPED
For the tasks of preprocessing
pandasdata and speeding up experiments, we have thePreprocessorclass and a number of base classes with single functionality at preprocessing.These methods should be implemented for
sparkdataframes, in the same paradigm as we have for theDesignerand theSplitter.At this moment, the implementation of the following methods is essential: