Skip to main content

5 docs tagged with "classification"

View all tags

Evaluation Metrics for Classification

Accuracy is the wrong metric more often than it is the right one. A model that predicts "healthy" for every patient in a dataset where 99% of patients are healthy scores 99% accuracy while being completely useless. Picking the right metric means picking it from the cost of each error type, not from convention.

k-Nearest Neighbors

Every model so far has fit parameters during training and thrown the raw data away afterward. kNN does the opposite: it stores the entire training set verbatim and defers all the actual work to inference time, where it asks "which stored examples look most like this new one?"

Logistic Regression

Despite the name, logistic regression is a classifier, and it remains the first model worth trying on any new tabular classification task — fast to train, easy to interpret, and a strong baseline against which everything fancier should be measured.

Naive Bayes

Naive Bayes assumes something that is almost never true — that every feature is independent given the class — and yet remains a strong baseline for text classification decades after more sophisticated methods appeared. Understanding why a false assumption still produces a useful classifier is the real lesson of this page.

Support Vector Machines

Logistic regression finds a separating line. Support vector machines find the separating line furthest from every training point — and it turns out only a handful of points (the "support vectors") actually determine where that line goes. The rest of the training set could be deleted without changing the model at all.