"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;

April 16, 2018

Day #105 - Ensemble Tips and Tricks

Diversity based on Algorithms
  • 2~3 gradient boosted trees (lightgb, xgboost, catboost)
  • Neural networks (Keras, Pytorch)
  • 1~2 Extra trees (Random Forest)
  • 1-2 knn models
Diversity based on inputdata
  • Categorical features (one hot, label encoding, target encoding)
  • Numerical features (Outliers, binning, derivatives, percentiles)
  • Interactions (col1*/+-col2),groupby,unsupervised
Subsequent level tips
  • GDM with depth 3
  • Linear models with high regularization
  • Extra trees
Happy Learning!!!
Post a Comment