Feature Selection Techniques
Created: =dateformat(this.file.ctime,"dd MMM yyyy, hh:mm a") | Modified: =dateformat(this.file.mtime,"dd MMM yyyy, hh:mm a")
Tags: knowledge
Filter Methods
- Univariate feature selection
- SelectKBest
- SelectPercentile
- SelectFpr, SelectFdr, or family wise error SelectFwe
- GenericUnivariateSelection
- Information gain
- Fischer score
- ANOVA F-Value for Feature Selection
- Correlation Matrix with Heatmap
Wrapper Methods
- Forward selection,
- iterative method in which we start with having no feature in the model. In each iteration, we keep adding the feature which best improves our model till an addition of a new variable does not improve the performance of the model.
- Backward elimination,
- inverse of forward. start with all the features and removes the least significant feature at each iteration which improves the performance of the model
- Exhaustive feature selection,
- the best subset of features is selected, over all possible feature subsets, by optimizing a specified performance metric for a certain machine learning algorithm
- greedy algorithm as it evaluates all possible feature combinations
- Recursive feature elimination.
- repeatedly creates models and keeps aside the best or the worst performing feature at each iteration. It constructs the next model with the left features until all the features are exhausted. It then ranks the features based on the order of their elimination.
- requires feature importance — MLP doesnt have by default, unless saliency
- Recursive feature elimination with cross-validation
Embedded Methods
- LASSO and RIDGE regression
- Random Forest Importance
MLP feature importance
- LIME, SHAP