6 docs tagged with "graphs"

A single kernel launch costs the CPU roughly 3–10 µs of driver-side work, independent of how much the kernel actually does. A pipeline that issues 50 small kernels per iteration can spend more time launching work than the GPU spends computing it, and Streams and Concurrency doesn't fix that — streams reorder and overlap launches, they don't reduce their count. A CUDA graph captures a whole sequence of operations once and replays it as a single launch, collapsing 50 dispatches into one.

A neural network is a function composition, layer feeding layer, and the graph of that composition is exactly what gets differentiated to train the network. Writing the forward pass explicitly as a graph of small, primitive operations is what turns "compute gradients" from a calculus exercise into a completely mechanical procedure — this page establishes that graph view before Backpropagation differentiates it.

Overview

Overview

Overview

Overview