[ИАД, осень 2025] Методы глубокого обучения. Семинар 3: CNN Implementation

Value of the Information & Strength of the Argument

The video provides substantial educational value by walking through multiple implementations of convolution, from naive to optimized, and explaining the underlying memory and performance considerations. The argumentation is solid: each step is justified with formulas, code, and benchmarks. The presenter clearly explains the trade-offs between different approaches, such as the speedup from using matrix multiplication and the importance of memory layout. The use of im2col is well-motivated, and the explanation of strides is thorough, making the content valuable for learners seeking a deep understanding of convolutional neural network internals.

Scientific Rigor, Source Quality, Title Accuracy

The scientific rigor is high: the presenter derives formulas, implements code, and validates results against PyTorch. The quality of sources is implicit, as the video is a course seminar; no external sources are cited, but the content is based on established deep learning principles. The title accurately reflects the content, which is a seminar on CNN implementation. The presenter’s occasional self-corrections (e.g., timing comparisons) demonstrate a careful approach. No comments were provided for analysis.

The title accurately reflects the content: a seminar on deep learning methods focusing on CNN implementation.

8/10

The video is a technical seminar that provides a rigorous, step-by-step implementation of convolution in Python, comparing naive, matrix-multiplication, and im2col approaches. The presenter demonstrates correctness through assertions and benchmarks against PyTorch, showing a strong methodological approach. However, the video is a seminar recording with limited production polish, and the presenter occasionally makes minor errors (e.g., in timing comparisons) that are corrected on the fly.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 00:00Introduction and overview of the seminar: implementing convolution from naive to optimized.
• 02:00Review of tensor formats (NHWC vs NCHW) and their implications.
• 05:00Derivation of the 2D convolution formula and extension to 4D.
• 10:00Naive implementation with nested loops; comparison to PyTorch (thousands of times slower).
• 15:00Convolution as matrix multiplication: 1x1 convolution and then general case with loops over kernel positions.
• 20:00Introduction to memory layouts (row-major vs column-major) and strides.
• 25:00Demonstration of stride-based operations: slicing, transposition, broadcasting, and their memory implications.
• 30:00Using as_strided to reshape matrices into tiles for efficient matrix multiplication.
• 35:00Introduction to im2col: extracting patches for convolution and preparing for matrix multiplication.
• 40:00Implementation of im2col-based convolution and performance comparison with PyTorch.

• PyTorch documentation — Referenced as the framework for the reference implementation and performance comparison.
• NumPy documentation — Used for array operations, strides, and as_strided function.

• PyTorch documentation — The reference implementation uses PyTorch's conv2d, which is consistent with the seminar's approach.
• NumPy documentation — The seminar relies on NumPy's as_strided and strides, which are documented.