CUDA: New Features and Beyond | NVIDIA GTC

CUDA: New Features and Beyond | NVIDIA GTC

by Stephen Jones

Applied Sciences & Engineering Computing & Cybersecurity UYFComputer architecture and logic designUYFPParallel processing
🎙 Stephen Jones 👥 222K 📅 March 31, 2026 ⏱ 44 min 👁 14K 📄 expert opinion 🧭 2026-08-13
Available in: English (current) Français

Keywords

CUDAGPUparallelismgreen contextsCUDA Tilemulti-nodeinference

Summary

Stephen Jones, Distinguished Software Architect at NVIDIA, presents the latest developments in CUDA and outlines future directions for GPU computing. He begins by contrasting symmetric and asymmetric parallelism, explaining how modern AI inference workloads benefit from disaggregated architectures where prefill and decode stages run on separate, appropriately configured resources. He introduces ‘green contexts’, a new CUDA feature that enables dynamic partitioning of a single GPU’s SMs, allowing fine-grained control over resource allocation within a single process. This enables patterns like low-latency reservation and dynamic partitioning, which are crucial for efficient inference. He then discusses the vision for multi-node CUDA, where CUDA graphs could span entire racks or data centers, requiring system-level solutions for naming, topology, and memory management. He also highlights CUDA Tile, a programming model extension for tensor operations, showing performance parity with hand-tuned kernels and portability across GPU architectures. The talk concludes with a look at ecosystem tools like Dynamo, GPU Direct Storage, and Nsight, emphasizing the importance of system software in enabling scalable GPU computing.

168 words

Critical Evaluation

Value of the Information & Strength of the Argument

The talk provides valuable insights into the evolution of CUDA, particularly the shift towards asymmetric parallelism and the introduction of green contexts for fine-grained resource control. The argumentation is solid, grounded in real-world inference workloads and performance data. The speaker clearly explains the motivation behind each feature and supports claims with benchmarks, such as the 90% performance parity of CUDA Tile kernels. The forward-looking discussion on multi-node CUDA is speculative but logically reasoned, identifying key challenges like naming and memory management. Overall, the information is highly relevant for developers and researchers in GPU computing.

Scientific Rigor, Source Quality, Title Accuracy

The presentation is technically rigorous, with the speaker demonstrating deep expertise. Sources are primarily internal NVIDIA projects (Dynamo, CUDA Tile, etc.) and references to talks by other NVIDIA engineers, which are credible within the industry. The title accurately reflects the content, covering both current features and future directions. The talk does not cite external academic sources, but this is typical for industry keynotes. The adéquation between title and content is excellent.

180 words

Title / Content Match

The title accurately reflects the content, which covers new CUDA features (green contexts, CUDA Tile) and future directions (multi-node CUDA).

Quality & Reliability

8/10

Presentation by a distinguished NVIDIA architect, covering recent CUDA features and future directions. Technical depth is high, but forward-looking statements are speculative and not peer-reviewed.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Cited Sources

  • NVIDIA Dynamo — Mentioned as the orchestrator for disaggregated inference workloads.
  • CUDA Tile — Announced as a new extension to the CUDA programming model for tensor operations.
  • GPU Direct Storage — Mentioned as a technology enabling low-latency transfers between GPU and storage.
  • NVIDIA Nsight — Mentioned as a set of developer tools for performance analysis.

Concurring Sources

Contribution & Novelties

The talk provides an insider’s perspective on NVIDIA’s roadmap for CUDA, particularly the introduction of green contexts for fine-grained GPU resource management and the vision for multi-node CUDA. It also showcases CUDA Tile’s performance and portability benefits. This is valuable for developers planning to leverage these features.

Pour aller plus loin :

88 words

Radar Profile

The radar profile shows high scores in technical depth and information quality, with slightly lower scores in accessibility and breadth. This indicates a technically dense presentation aimed at an expert audience, with strong focus on specific CUDA features and future directions.

Reliability 8/10