Images as Tensors
Before any model sees a pixel, an image is a grid of numbers — and the conventions for arranging and scaling those numbers disagree between nearly every library you'll touch. Almost every mysterious vision bug traces back to one of these conventions being silently wrong: a colour channel swapped, a value range mismatched, an axis order flipped.
Convolution Operation
A fully-connected layer applied directly to a 224×224 RGB image would need over 150,000 input weights per single output unit — and that's before considering how many units a layer needs. Convolution replaced that with a small, shared filter slid across the image, cutting parameters by orders of magnitude while adding a property fully-connected layers structurally lack: the same filter finds the same pattern no matter where in the image it appears.
Pooling & Shape Arithmetic
A CNN's spatial resolution has to shrink somewhere between a 224×224 input and a single classification decision — pooling is the classic way to do that shrinking, trading spatial precision for a degree of invariance and reduced compute. Getting the resulting shapes right, at every layer, is the single most useful bookkeeping habit for building a CNN that actually runs.
CNN Architectures
Fifteen years of vision architecture research boiled down to a handful of ideas that survived contact with reality: go deeper, but only once you can actually train the depth; use small filters repeatedly rather than large ones once; and share computation aggressively when compute or memory is scarce. Nearly every architecture below is one of these ideas, applied and refined.
Data Augmentation
The cheapest way to get more training data isn't collecting more — it's transforming the data you already have in ways that shouldn't change the label. A flipped photo of a cat is still a cat; a rotated photo of a stop sign is still a stop sign (probably). What you choose to augment with is a direct statement of what invariances you want the model to learn.
Transfer Learning for Vision
Almost nobody trains a vision model from scratch anymore, and there's a good structural reason: the early layers of any CNN trained on natural images learn something close to universal — edges, colours, simple textures — regardless of what specific objects the model was ultimately trained to recognise. That generic foundation is exactly what transfer learning reuses.
Object Detection
Classification answers "what is in this image?" Detection answers a harder question: what is in this image, where exactly is it, and how many are there? That extra "where" and "how many" is precisely what a plain classifier cannot provide, and the entire design space of detection architectures is really about different ways to propose candidate locations.
Segmentation
Object Detection draws a rectangle around an object. Segmentation is more demanding still: classify every single pixel. The architecture this demands has a distinctive shape — downsample to understand what's in the image, then upsample back to full resolution to say exactly where — and the skip connections carrying detail across that shape are the entire design.
Vision Transformers
The architecture that took over NLP turns out to work for images too, once you accept one reframing: an image is a sequence of patches. Once that step is taken, the entire transformer stack from Transformer Architecture transfers to vision essentially unchanged — the same attention, the same feed-forward blocks, the same residual and normalisation pattern.
Self-Supervised Vision
Labelled image data is expensive; unlabelled photos are nearly free. Self-supervised vision learns useful visual features from the unlabelled kind alone, by manufacturing a training signal directly from the image itself — the vision-side counterpart to Pretraining Objectives's masked and causal language modelling.
Multimodal Vision-Language
An image and the sentence describing it are, on the surface, completely different kinds of data — a grid of pixels versus a sequence of tokens. CLIP's contribution was to train both an image encoder and a text encoder so that matching pairs land in the same embedding space, close together — and the moment that works, classification becomes a search problem, with no fixed label set required.
CNN Interpretability
A model that's 98% accurate on your test set can still be completely wrong about why. A famous, real example: a classifier trained to distinguish huskies from wolves turned out to be detecting snow in the background, not the animal — accurate on a test set that happened to reflect the same correlation, and silently broken the moment that correlation didn't hold. Interpretability tools exist to catch exactly this before deployment, not after.
Deploying Vision Models
The model scores 95% accuracy in the notebook — and returns nonsense in production. This is one of the most common, most preventable failure modes in applied vision, and it's almost never the model's fault: it's a mismatch between how the training pipeline preprocessed images and how the serving pipeline does.