Gradient Boosting algorithms Machine Learning Algorithm

Machine Learning / Gradient Boosting algorithms Machine Learning Algorithm

GBM combines the predictions from multiple decision trees to generate the final predictions. Decision trees errors or mistakes from previous are recovered in the next decision tree.

Decision Tree1------>Decision Tree2------>Decision Tree3------>Decision Treen

On the input data set used to generate a Decision tree on which predictions are applied.

Data	Data	Data	Data	Data
Decision tree1	Decision tree2	Decision tree3	------	Decision treen
Prediction	Prediction	Prediction	Prediction	Prediction

Machine learning algorithms errors are of 2 types. They were

1. Bias Error.
2. Variance Error.

Gradient Boosting algorithms are used to minimize bias error of the model.

Bias error
It is error caused due to the simplified assumptions used in a model for easier estimation of target functions

Variance
It is the Change of estimate amount of the target function when different training data was used

Extreme Gradient Boosting Machine (XGBM)
It is a modified version of Gradient Boosting Machine (GBM) but implements parallel preprocessing (at the node level). LightGBM
It concentrates on the leaf wise growth but not level wise growth and used for large amounts of data. It is very fast and efficient.

CatBoost
It is used to deal with categorical variables in the data. Categorical variables are convert to numerical ones are done by statistics on combinations of features.

Home Back