Skip to content

[ENH] Specialised preprocessing techniques for change and anomaly detection #28

Description

@Tveten

Most methods in skchange require the data to be preprocessed in order for default hyperparameter settings to be appropriate. The package should come with a few standard preprocessing techniques to be more easily applied correctly. Suggestions:

  • Centering by the median and scaling by the median absolute deviation of the data. This is a common preprocessing technique for anomaly detection.
  • Centering by the median and scaling by 1/sqrt(2) times the median absolute deviation of the differenced data. This is a common preprocessing technique for changepoint detection. Rationale: For data that are iid within each segment, $Var[x_t - x_{t-1}] = Var[x_t] + Var[x_{t-1}] = 2Var[X_t]$. Thus, $Var[X_t] = Var[x_t - x_{t-1}]/2$. In the differenced data, the chanepoints will appear as outliers, so the variance/std should be estimated robustly, e.g. by the median absolute deviation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementautolabel: New feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions