Learning Paradigms

Before picking an algorithm, answer one question: what does the training signal look like? A dataset of (input, correct-output) pairs calls for a different family of methods than a pile of unlabelled data, which in turn differs from a system that only gets a delayed reward for a sequence of actions. The paradigm is chosen by the data you have, not by which algorithm sounds most impressive.

Key idea

Paradigms are classified by what supervision is available, not by what model is used — a neural network can be trained under any of them.

Three panels: labelled points with a boundary, unlabelled points with discovered groups, and a trajectory collecting rewards — The three paradigms on the same scatter of points. Supervised learning is told the answer; unsupervised must find structure; reinforcement learning only ever sees how good the outcome was.

Supervised learning

Every training example comes with the "correct answer." The model learns a function $f: X \to Y$ from labelled pairs $(x_i, y_i)$ .

Regression — $Y$ is continuous (predict a house price, a temperature).
Classification — $Y$ is categorical (predict spam/not-spam, a digit 0–9).

Formally, supervised learning searches for the $f$ minimising expected loss over the true (unknown) data distribution:

f^* = \arg\min_f \; \mathbb{E}_{(x,y) \sim \mathcal{D}}\big[L(f(x), y)\big]

Symbol	Meaning
$X, Y$	input space and output space
$f$	the function being learned, mapping $X \to Y$
$L$	the loss function penalising a wrong prediction
$\mathcal{D}$	the true, unknown distribution generating $(x, y)$ pairs
$\mathbb{E}$	expectation — the average loss if you could see every possible $(x,y)$

You never see $\mathcal{D}$ directly — you only get a finite sample from it, which is why generalisation (covered in Bias-Variance Tradeoff) is the central concern of the whole field.

Unsupervised learning

No labels at all — only $x_i$ , no $y_i$ . The goal shifts from "predict the output" to "describe the structure":

Clustering — group similar points (customer segments, document topics).
Density estimation — model $p(x)$ , the probability of observing a given input.
Dimensionality reduction — find a lower-dimensional representation that preserves what matters.

Self-supervised learning

A middle path: manufacture labels from the unlabelled data itself. Mask a word and predict it from context; predict the next frame of a video; rotate an image and predict the rotation. No human ever labelled anything, but the training loop looks exactly like supervised learning because the "label" is a piece of the same data held out. This is the mechanism behind every modern large language model — see Pretraining Objectives.

Semi-supervised learning

A small labelled set plus a large unlabelled set. Common when labelling is expensive (medical images) but raw data is cheap. Techniques typically use the unlabelled data to shape the model's notion of "plausible input" and the labelled data to pin down the actual task.

Reinforcement learning

No fixed dataset at all — an agent takes actions in an environment and receives a reward signal, often delayed and sparse. The training data is generated by the agent's own behaviour, which makes this qualitatively different from every paradigm above. Covered in full in Reinforcement Learning.

Decision table

You have...	Reach for
Labelled examples, continuous target	Supervised regression
Labelled examples, categorical target	Supervised classification
Only unlabelled data, want groups	Clustering
Only unlabelled data, want a smaller representation	Dimensionality reduction
Huge unlabelled corpus, want a general-purpose model	Self-supervised pretraining
A little labelled data, a lot of unlabelled data	Semi-supervised learning
An environment, a reward signal, no fixed dataset	Reinforcement learning

Where each is covered

Supervised learning: Classical ML and Deep Learning.
Unsupervised learning: k-Means Clustering, PCA and SVD.
Self-supervised learning: Pretraining Objectives, Self-Supervised Vision.
Reinforcement learning: the entire Reinforcement Learning section.

Code: one example per paradigm

paradigms_demo.py
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.cluster import KMeans

X, y = load_iris(return_X_y=True)

# --- Supervised: a classifier trained on (X, y) pairs ---
clf = LogisticRegression(max_iter=200).fit(X, y)
print("supervised accuracy:", clf.score(X, y))

# --- Unsupervised: k-means sees only X, no labels ---
km = KMeans(n_clusters=3, n_init=10, random_state=0).fit(X)
print("unsupervised cluster sizes:", np.bincount(km.labels_))

# --- Toy self-supervised objective: predict a masked feature from the rest ---
masked_col = 0
X_context = np.delete(X, masked_col, axis=1)
X_target = X[:, masked_col]
from sklearn.linear_model import LinearRegression
ssl_model = LinearRegression().fit(X_context, X_target)
print("self-supervised R^2 predicting masked feature:", ssl_model.score(X_context, X_target))

The self-supervised block is the important one: X_target was never a human-provided label, it is a column of X we deliberately hid and asked the model to reconstruct from the rest — the exact trick behind masked language modelling.

Supervised learning​

Unsupervised learning​

Self-supervised learning​

Semi-supervised learning​

Reinforcement learning​

Decision table​

Where each is covered​

Code: one example per paradigm​

See also​