The non-linearity between layers is what makes depth meaningful at all — From Perceptron to MLP proved that a stack of purely linear layers collapses into a single linear layer. But not all non-linearities are equal: the shape of an activation's derivative determines whether gradients survive a deep network or vanish before reaching the early layers.
A spec sheet lists a GPU as a pile of numbers — core count, clock speed, memory bandwidth — but those numbers only make sense once you know what physical structure they're describing. A GPU is not a bag of independent processors; it's a small number of large, warp-scheduling processors (streaming multiprocessors), each built from smaller replicated pieces, all sharing a common path out to memory. Understanding that structure top to bottom is what turns a spec sheet from marketing copy into something you can reason about.
Instead of compressing the entire input into one fixed vector and hoping nothing important got lost, attention lets the decoder look back at every encoder position directly, every time it generates a token, and decide for itself which parts of the input actually matter right now. It's a differentiable lookup — a weighted average, where the weights are learned rather than fixed.
Every doc in this section assumes a specific split: systems and data live in C++; composition and
Overview
Three decisions explain almost every API in spdlog: formatting is delegated to fmt, sinks are the
Most projects start single-player, or single-player-first with multiplayer as a maybe-someday. That's a
Every later doc in this section assumes you can place a new piece of knowledge somewhere on a map.
Why this matters
Plain RNNs forget almost everything within a few dozen timesteps — the vanishing-gradient product from Recurrent Neural Networks sees to that. In 1997, long before deep learning was mainstream, a fix was published that kept recurrent networks the dominant sequence architecture for another two decades: give the gradient an additive path through time, gated by learned switches deciding what to keep and what to forget.
LangChain's packages form a dependency tree. Understanding the direction of that tree tells you which package to import an abstraction from, and which packages are safe to depend on for the long term.
Every network up to this point has processed a single fixed-size input. Language, audio, and time series don't come in fixed sizes — a sentence can be five words or fifty. Recurrent networks were the first architecture built specifically to handle that: reuse the same weights at every timestep, carrying a hidden state forward as a compressed summary of everything seen so far.
Everything in this folder so far has described one architecture. The reason that is a reasonable way to spend eleven pages is that Cortex-M is what the overwhelming majority of microcontroller work is written against. The reason it is not the only thing worth knowing is that RISC-V parts are now genuinely shipping in volume — the ESP32-C3 in a hobbyist's hands, the CH32V003 at ten cents, the RISC-V management cores inside SoCs whose application processors are Arm — and the transition is much easier than a new instruction set sounds, provided you know which of your Cortex-M assumptions are architectural and which are Arm's.
The split of responsibility is deliberate: a logger decides whether to log at all, a sink decides
In 2015, researchers found something strange: a 56-layer network had higher training error than a 20-layer network on the same task — not overfitting (that would show as a validation gap), but a genuine failure to optimise the deeper network at all. The fix, adding the input back to a layer's output, was a two-line change that took feasible network depth from roughly twenty layers to over a thousand.
Arm does not sell chips. It sells processor designs, and a silicon vendor — ST, NXP, Nordic, Raspberry Pi — licenses one, wraps it in memory and peripherals, and sells you the result. That arrangement is why "it's an Arm chip" tells you almost nothing on its own, and why the useful question is always which Arm core, in which configuration, from which vendor.
The SM is the unit everything about GPU performance is ultimately accounted against: occupancy, register pressure, shared-memory capacity, and warp scheduling are all per-SM quantities. Zooming into one SM explains why a block, once scheduled, stays resident on a single SM for its entire lifetime, and why the resources that limit how many blocks can run concurrently are the ones this page enumerates.
The 2017 paper that introduced the transformer had a blunt thesis: attention was the useful part of the encoder-decoder architecture, so delete everything else. No recurrence, no convolution — just attention and simple feedforward layers, stacked. Removing the sequential dependency of recurrence is what turned scale from a research curiosity into an engineering problem that money and hardware could actually solve.
One architecture, three ways to cut it — and which cut you choose determines what the resulting model can actually do. The difference between a model that reads and a model that writes turns out to come down to one thing: the shape of the attention mask.
Every accelerator in this section so far has been a variation on "more programmable cores, running in parallel." An NPU (neural processing unit) is a different move entirely: instead of adding parallel general-purpose lanes, it removes almost all of the general-purpose machinery and replaces it with a dataflow of multiply-accumulate (MAC) cells wired specifically for tensor arithmetic. CPU vs GPU vs NPU already introduced this as the third design point; this page works through what that design point actually buys and what it costs.
A framework earns its keep when it removes real complexity. For the simplest LLM use case — one
The useful question is not "why is a GPU faster than a CPU" — it usually isn't. A single CPU core will finish one dependent chain of instructions sooner than any GPU will, and it will do it on branchy, pointer-chasing, irregular code that a GPU handles badly. The real question is how a fixed transistor budget gets spent. A CPU spends most of its area on machinery that makes one instruction stream go fast: out-of-order scheduling, register renaming, branch prediction, and a deep cache hierarchy that hides DRAM latency from a handful of threads. A GPU deletes almost all of that and spends the reclaimed area on arithmetic units, then keeps them busy by oversubscribing the machine with far more threads than can execute in any one cycle.