10.6: Conclusion

For creating a machine learning model, no guaranteed method works. This is especially true for text classification models. Using a variety of feature engineering techniques can improve the odds of being able to create models with high predictive performance. In the case of text classification, features are in thousands. More the features, the more the model complexity. Complex models lead to higher computational costs to train and use such models. Feature selection can help us reduce the number of features and model complexity. We can also use feature reduction to reduce the size of the feature matrix, without losing information from features. This can reduce model complexity even further.

Many times, we find different modeling techniques perform differently on the same test data. To harness the power of different models, we can ensemble. The drawback of ensembling is that it will lead to a huge number of base models. This results in higher computational costs for training and using the model.

Ensemble feature selection can help solve all of the above problems by doing feature selection for each base model to identify fewer complex models for different combinations of features, feature representation, and models. Finally, it identifies the best set of models which give the highest predictive performance. Thereby reducing the number of base models.