7.7: SVM

Support vector machines learn very well from high-dimensional data. When we have more features, SVM uses the extra information from additional features and tries to build a better model. This is true even when multicollinearity exists. Hence, feature selection doesn't help mostly in improving performance for SVM.

If overfitting is of concern, we can instead select a suitable value of SVM parameter C through experimentation. It intuitively acts like ridge regression. In that it doesn't remove variables. It only changes the level of impact a specific feature has. This is best suited for situations when the feature set has many binary features, which increases degrees of freedom.