17 docs tagged with "performance"

Every kernel is limited by one of two things before it is limited by anything else: how fast the device can do arithmetic, or how fast the device can move bytes from DRAM. Which one applies is a property of the kernel's own math, computable on paper before you write a line of CUDA, and it determines almost everything about how you should spend optimization effort afterward. The roofline model is the tool that turns "which one applies" into a single number you can compute and a single plot you can place it on.

Every Runnable exposes sync and async twins (invoke/ainvoke, batch/abatch, stream/astream, from Runnables and LCEL), plus batch for running many independent inputs efficiently.

Async logging is not strictly better than sync — it trades tail latency and complexity for

"Blueprint is slow" is the kind of claim that's technically true and mostly useless. It's true in the

A runtime level check is cheap, but it isn't free — it's still a branch and, on the disabled path,

Copy creates a duplicate of an object. Move transfers ownership of resources from one object to another. Understanding when each happens is crucial for performance and correctness.

Writing float in C on a microcontroller does not tell you what the hardware will do. The same line of source can compile to one instruction, to a forty-cycle library call, or to a two-hundred-cycle double-precision emulation — and the compiler chooses silently, based on flags you may not have set deliberately. Nothing in the source distinguishes the three cases, which is why "why is my control loop suddenly missing deadlines" is so often a floating-point question.

The fastest way to make training slower is to leave the GPU waiting on the CPU — and the second fastest way to make it faster, after fixing that, is to stop computing every number with more precision than the task actually needs. Most real speedups come from these two unglamorous facts, not from a cleverer algorithm.

Making the model cheap enough to serve, without quietly making it wrong. Every optimisation technique on this page trades accuracy, latency, or memory for one another — measure all three, before and after, or the "optimisation" is a guess.

inline suggests the compiler replace function calls with function body, eliminating call overhead. Modern compilers decide automatically.

Data arranged at addresses that are multiples of its size. Required for correctness on some architectures, critical for performance on all.

Knowing a kernel's arithmetic intensity puts it on the roofline plot in theory; knowing whether it is actually memory-bound or compute-bound in practice requires measuring the running kernel, because achieved bandwidth and achieved compute throughput are never the datasheet peaks the paper estimate assumed. This page turns the roofline classification from Arithmetic Intensity and the Roofline Model into a concrete diagnostic you run against a profiler, and adds the case the roofline model doesn't represent at all.

The library's defaults are tuned for ergonomics, not throughput, and most of the performance

fmt is fast for structural reasons, not micro-optimizations — knowing which ones lets you tell when

Tools for finding performance bottlenecks: CPU time, cache misses, branch mispredictions. Measure first, then optimize.

Understanding compiler-generated assembly helps verify optimizations, debug performance issues, and understand low-level behavior.

Why this matters