Atomic Operations
Some updates can't wait for a barrier — a histogram bin, a running total, a lock-free counter — because the threads touching the same location aren't at a point where syncthreads() or a group sync() even applies; they need the read-modify-write itself to be indivisible. Atomics provide that: a hardware-guaranteed sequence of read, modify, and write on a single memory location that no other thread's atomic on the same location can interleave with. What atomics don't provide is speed for free — how many threads target the same address, not how many threads issue atomics in total, is what determines whether that guarantee is nearly free or a serialization bottleneck.