Day #105 - Ensemble Tips and Tricks

Diversity based on Algorithms

• 2~3 gradient boosted trees (lightgb, xgboost, catboost)
• Neural networks (Keras, Pytorch)
• 1~2 Extra trees (Random Forest)
• 1-2 knn models

Diversity based on inputdata

• Categorical features (one hot, label encoding, target encoding)
• Numerical features (Outliers, binning, derivatives, percentiles)
• Interactions (col1*/+-col2),groupby,unsupervised

Subsequent level tips

• GDM with depth 3
• Linear models with high regularization
• Extra trees

Happy Learning!!!