6.6: Conclusion

Filter method can be very useful if we have huge number of features, say thousands, and we want to try the first model. We want to ascertain how well the model can perform. But we have limited computing power to train the model with all the features. Filter methods can help us reduce the size of feature matrix. This comes with a tradeoff between losing an important predictor. Something we should keep in mind while performing feature selection with filter method.

Wrapper methods is computationally expensive. The only exception to this is the forward selection method. It may or may not be useful in searching better performing models. This method should be tried if we have the required computing power and time to explore a greater number of feature selection methods.