Drift Detection and ML Solution Retraining (4/4)

This video is the fourth part of a series on drift detection and model retraining. The speaker, Samuel Ackerman from IBM, begins by recapping previous discussions on using statistical tests to detect distributional drift. He then focuses on distance metrics for categorical data, highlighting the asymmetry of chi-square tests and introducing the Jensen-Shannon distance as a symmetric alternative. He explains its properties, including boundedness and handling of zero-frequency categories. He also discusses using slice-based intermediate representations for drift detection, where slices are hyper-rectangles in feature space with higher-than-average error rates. The method involves comparing the proportion of observations falling into each slice across two datasets, using hypothesis testing and multiple comparisons correction. The talk is interactive, with questions from the audience, and ends with a promise to continue in the next session.