Regression Trees

Item: Regression Trees
Rating: 4
Author: PostSingularity Institute

by Machine Learning Practice

Formal & Physical Sciences AI — Models & Research UYQArtificial intelligence (AI)UYQMMachine learning

🎙 Machine Learning Practice 👥 419 📅 October 26, 2022 ⏱ 27 min 👁 100 📄 tutorial 🧭 2026-08-17

Available in: English (current) Français

Keywords

regression treedecision treemean squared errorpiecewise constantgreedy algorithm

Summary

This tutorial explains the concept of regression trees, a type of decision tree that outputs continuous values rather than classes. The video begins by contrasting regression trees with classification trees, highlighting that regression trees can output a constant value per leaf (piecewise constant function) or a more complex function. The mathematical formulation is presented, focusing on the mean squared error (MSE) as the cost function. The derivation shows that the optimal constant for each leaf is the average of the target values in that leaf. The video then discusses how to evaluate potential splits by computing the reduction in MSE, leading to a greedy algorithm for tree growth. A simple example is drawn to illustrate the process of splitting and refining the tree. The video concludes by mentioning that the next video will cover coding the algorithm.

137 words

Critical Evaluation

Value of the Information & Strength of the Argument

The video provides a solid introduction to regression trees, with clear mathematical derivations and a step-by-step example. The argumentation is logical and builds from basic definitions to the greedy splitting criterion. The value lies in its pedagogical clarity, making complex concepts accessible. However, it lacks depth in discussing regularization and overfitting, and does not cover advanced topics like pruning or handling categorical features.

Scientific Rigor, Source Quality, Title Accuracy

The video is a tutorial with no external sources cited, which is typical for such content. The mathematical content is accurate and well-presented, but the lack of references to literature or further reading is a limitation. The title accurately reflects the content, and the video stays on topic throughout.

127 words

Title / Content Match

The title accurately reflects the content, which focuses exclusively on regression trees.

Quality & Reliability

7/10

The video provides a clear mathematical derivation of regression trees, including the cost function and split improvement. It is a tutorial with no external sources cited, but the content is accurate and well-explained. The lack of references and the informal presentation style slightly reduce the score.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Contribution & Novelties

The video provides a clear and concise explanation of regression trees, with a focus on the mathematical foundations. It is a good starting point for learners. For further exploration, consider the following:

Decision tree learning - Wikipedia — Overview of decision trees, including regression trees.
Classification and regression trees - Wikipedia — General information on CART.
Mean squared error - Wikipedia — Definition and properties of MSE.
Greedy algorithm - Wikipedia — Explanation of greedy algorithms used in tree building.
Pruning (decision trees) - Wikipedia — Techniques to reduce overfitting in decision trees.

93 words

Radar Profile

The radar profile shows balanced scores across all dimensions, with slightly higher quality of information and technical level, indicating a solid educational resource. The lower quantity of information and global reliability reflect the lack of external references and limited scope.

Reliability 7/10