Sequential Drift Detection

Sequential Drift Detection

by Samuel Ackerman

Formal & Physical Sciences AI — Models & Research UYQArtificial intelligence (AI)UYQMMachine learning
🎙 Samuel Ackerman 👥 46 📅 August 18, 2022 ⏱ 34 min 👁 10 📄 expert opinion 🧭 2026-08-18
Available in: English (current) Français

Keywords

drift detectionconcept driftsequential analysischange point detectiononline learning

Summary

In this talk, Samuel Ackerman from IBM provides an overview of sequential drift detection, a method for detecting changes in data distributions over time. He contrasts it with two-sample drift detection, where only two datasets are compared. Sequential drift detection involves a series of datasets ordered in time, and the goal is to detect when the distribution changes. He discusses supervised vs. unsupervised approaches, with unsupervised being more common. He outlines key aspects of drift detection algorithms: parametric vs. non-parametric, memory and windowing, online vs. offline, and whether they detect single or multiple change points. He mentions specific algorithms such as ADWIN, HDDM, DDM, and CPM, and libraries like River and Ruptures. He highlights the importance of false positive control in sequential testing, noting that many algorithms do not account for multiple testing, but CPM does. He concludes with a Q&A session where he clarifies the difference between sequential and repeated two-sample tests, emphasizing that drift is about a change point affecting subsequent data.

164 words

Critical Evaluation

Value of the Information & Strength of the Argument

The talk provides a valuable overview of drift detection methods, covering key concepts and practical considerations. The speaker’s experience is evident, and he offers insights into the strengths and weaknesses of various algorithms. The argumentation is clear and logical, though it lacks formal proofs or detailed comparisons. The discussion of false positive control is particularly valuable, as it highlights a common pitfall in sequential testing.

Scientific Rigor, Source Quality, Title Accuracy

The talk is based on the speaker’s practical experience and mentions several algorithms and libraries, but does not provide formal citations. The title accurately reflects the content. The speaker does not delve into mathematical details, but the information is presented in a structured manner. The lack of formal references reduces the scientific rigor, but the practical insights are still valuable.

140 words

Title / Content Match

The title accurately reflects the content, which focuses on sequential drift detection methods.

Quality & Reliability

7/10

The speaker is a practitioner from IBM, providing an overview of drift detection methods. The content is based on practical experience and mentions several algorithms and libraries, but lacks formal citations and detailed mathematical derivations.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Cited Sources

  • River — Mentioned as a Python library implementing drift detection methods.
  • Ruptures — Mentioned as a library for change point detection.
  • CPM (Change Point Model) — Mentioned as an R package with false positive control.

Concurring Sources

  • River documentation — Provides implementations of drift detection methods mentioned in the talk.
  • Ruptures documentation — Provides tools for change point detection, consistent with the talk's description.

Contribution & Novelties

The talk provides a practical overview of drift detection methods, highlighting the importance of false positive control in sequential testing. It offers insights into algorithm selection and implementation, based on the speaker’s experience.

Pour aller plus loin :

  • Concept Drift — Overview of concept drift in machine learning.
  • ADWIN — Original paper on ADWIN algorithm.
  • Change Point Detection — General overview of change point detection.

65 words

Radar Profile

The radar profile shows balanced scores across all dimensions, with slightly higher scores in information quantity and quality, indicating a solid overview with practical insights. The technical level is moderate, making it accessible to a broad audience.

Reliability 7/10