Curse of dimensionality

In this lecture, Dr. Eitan Farchi explains the curse of dimensionality, focusing on the counterintuitive fact that the volume of a unit sphere tends to zero as the number of dimensions increases. He begins by recalling the previous discussion and then presents a Monte Carlo simulation to estimate the volume of a sphere for dimensions from 1 to 20. The simulation randomly samples points from a Gaussian distribution and counts how many fall within the unit sphere. The results show that the estimated volume decreases rapidly with increasing dimensions, confirming the theoretical prediction. The speaker then discusses the implications for machine learning, noting that in high-dimensional spaces, data points tend to be far apart, making distance-based methods like k-nearest neighbors less effective. He also draws analogies to relativity and quantum theory, where our intuition fails due to lack of experience. The lecture concludes with a brief Q&A session, addressing questions about the intuition behind the phenomenon and its relevance to adversarial attacks.