7 docs tagged with "synchronization"

The warp-level tools covered so far — divergence handling, independent thread scheduling, the shuffle and vote intrinsics — all operate within a single warp of 32 threads. Most kernels that use shared memory need something coarser: a guarantee that every thread in the whole block, potentially many warps, has reached a point and that everything they wrote before that point is visible to everything they read after it. That guarantee is syncthreads(), and getting its rules exactly right is what stands between a working tiled kernel and one that hangs or reads garbage on some inputs and not others.

The C++ memory model defines how threads interact through memory, including visibility of writes, ordering of operations, and synchronization primitives.

Condition variables allow threads to wait for specific conditions to become true, enabling efficient thread coordination beyond simple mutexes.

Warp-Level Primitives and Block Synchronization both work, but both lean on implicit context there is no implicit lockstep assumption left to break, because every operation states which threads it applies to.

A data race occurs when two or more threads access the same memory location concurrently, at least one access is a write, and there's no synchronization between them. Data races cause undefined behavior.

syncthreads() barriers a block; cluster.sync() barriers a cluster; neither reaches every block in a grid. Some algorithms genuinely need that — a multi-pass iterative solver that must finish writing generation *N* everywhere before any block reads generation *N* for generation *N+1*, for instance — and the usual answer, launching a second kernel between the passes, has real cost when the intermediate state is large and expensive to leave and re-establish. Grid-wide synchronization exists for that case, but it is not simply "a bigger syncthreads()": it comes with a hardware constraint that shapes the whole launch around it.

A mutex (mutual exclusion) is a synchronization primitive that protects shared data by allowing only one thread to access it at a time.