2 docs tagged with "boosting"

This page uses xgboost, lightgbm, and catboost — none is in the knowledge base's default library set (numpy, scipy, pandas, matplotlib, scikit-learn, torch, torchvision, transformers, datasets). They are the industry-standard implementations of Gradient Boosting and this page cannot honestly cover the topic without them. Install with pip install xgboost lightgbm catboost.

Random forests average many independent trees to cancel out variance. Gradient boosting does something structurally different: it builds trees one at a time, each new tree specifically targeting the mistakes the ensemble has made so far. Where bagging reduces variance, boosting reduces bias — and the combination of the two ideas covers most of what wins tabular ML competitions.