2 docs tagged with "svm"

Some data simply cannot be separated by a straight line — two concentric circles of different classes have no linear boundary at all. The kernel trick lets a linear algorithm operate as if the data had been lifted into a much higher-dimensional (sometimes infinite-dimensional) space where a linear boundary does exist, without ever actually computing that lift.

Logistic regression finds a separating line. Support vector machines find the separating line furthest from every training point — and it turns out only a handful of points (the "support vectors") actually determine where that line goes. The rest of the training set could be deleted without changing the model at all.