Feature selection and model interpretability

In short

Feature selection is the task of choosing a small, useful subset of input variables from a larger set, and model interpretability is the task of explaining what a trained model has learned and why it made a particular prediction. Both attack the same practical problem: a model with 500 noisy inputs is slower, overfits more, costs more to serve and cannot be defended to a regulator or a business user. University syllabi examine filter/wrapper/embedded methods, the information-theoretic measures behind them, regularisation-based selection and the modern explanation tools (SHAP, LIME, PDP, permutation importance); GATE DA asks numerical questions on mutual information, chi-square and information gain. In Indian industry this is everyday work in credit scoring and fraud at HDFC, ICICI, Bajaj Finserv, Paytm and Razorpay, where RBI expects lenders to explain adverse decisions, and in healthcare and insurance analytics; interviewers almost always ask "which features mattered and how do you know?"

Explained

Why select features at all

• Curse of dimensionality: as dimensions grow, data becomes sparse, distances lose contrast, and the sample size needed for a given accuracy rises steeply - k-NN, SVM with RBF kernels and clustering suffer most.
• Overfitting: irrelevant features let the model fit noise; variance rises while bias barely falls.
• Cost and latency: fewer features mean smaller feature pipelines, cheaper storage, faster inference - a real constraint at UPI-scale fraud scoring (thousands of decisions per second).
• Interpretability and compliance: a 12-feature scorecard can be explained to a customer or an auditor; a 900-feature gradient-boosted model cannot without extra tooling.
• Data quality: dropping features removes sources of leakage, drift and missingness.
• Feature selection vs feature extraction: selection keeps a subset of the *original* columns (names preserved, interpretable); extraction (PCA, autoencoders) builds new combined axes - interpretability is lost but variance is compressed.
• Relevance vs redundancy: a good subset is relevant to the target *and* non-redundant among itself. Two perfectly correlated predictors are individually relevant but jointly wasteful.
• Feature categories: strongly relevant (removing it hurts), weakly relevant (useful only with others, e.g. XOR features), irrelevant, redundant.

Filter methods (model-independent, run before training)

• Score each feature against the target with a statistic, rank, keep the top-k or those above a threshold. Fast, scalable, no model needed; ignore feature interactions and the downstream model.
• Variance threshold: drop near-constant columns (unsupervised). A feature that is 99.5% "0" rarely helps.
• Pearson correlation |r|: numeric feature vs numeric target; also used feature-vs-feature to kill redundancy.
• Chi-square test: categorical feature vs categorical target on a contingency table of counts; needs non-negative counts, not standardised values.
• ANOVA F-test (f_classif): numeric feature, categorical target - ratio of between-group to within-group variance.
• Mutual information: captures any dependence, linear or not; I(X;Y) = 0 only if independent. Preferred over correlation for non-monotonic relations.
• Information gain / Gini decrease on a single split - the same quantity decision trees use.
• Fisher score, ReliefF: distance-based relevance measures used in older syllabi.
• mRMR (minimum redundancy maximum relevance): maximise relevance to target minus average mutual information with already-selected features - the standard answer for "how do filters handle redundancy?"
• Multicollinearity check - VIF: VIF_j = 1/(1 − R²_j) from regressing feature j on the others. VIF above 5 (strict) or 10 (common rule) signals redundancy; drop or combine. Essential for linear/logistic coefficients and for credit scorecards.

Wrapper methods (use the model's own score)

• Treat selection as a search over subsets, scoring each subset by cross-validated model performance. Accurate for the chosen model, but expensive - 2^p subsets in full.
• Forward selection: start empty; repeatedly add the feature giving the biggest CV gain; stop when the gain is below a threshold.
• Backward elimination: start with all; repeatedly drop the least useful feature. Needs n > p.
• Stepwise (bidirectional): add and remove alternately, so an early mistake can be undone.
• Recursive Feature Elimination (RFE): fit model, rank by coefficient magnitude or importance, drop the weakest, refit; repeat. RFECV picks the subset size by cross-validation.
• Exhaustive search: all subsets - only for very small p.
• Genetic algorithms / simulated annealing: stochastic subset search for large p.
• Risk: selecting on the same data used to evaluate leaks information and inflates scores - the selection step must sit *inside* the CV loop (see Common mistakes).

Embedded methods (selection happens during training)

• L1 / Lasso: the absolute-value penalty has corners at zero, so increasing λ drives coefficients exactly to zero - selection and fitting in one step. L2 (ridge) shrinks but never zeroes, so it does not select.
• Elastic Net: αL1 + (1−α)L2; keeps L1's sparsity while handling groups of correlated features more stably.
• Tree-based importance: total impurity decrease (Gini importance / MDI) attributable to each feature across the forest. Fast but biased towards high-cardinality and continuous features, and computed on training data.
• Gradient boosting importances: gain, cover, weight (frequency) in XGBoost/LightGBM - gain is usually the one to quote.
• Boruta: compares each real feature's importance against "shadow" features made by shuffling columns; keeps only those that significantly beat the best shadow.
• Regularised linear SVM, group lasso (drops whole groups of dummy columns together).
• Stability selection: repeat Lasso on bootstrap samples; keep features selected in a high fraction of runs.

Interpretability: concepts and intrinsic models

• Interpretability - how readily a human can follow the model's mechanism; explainability - the ability to produce a reason for an output, often via a separate tool. Transparency usually means the mechanism is open to inspection.
• Global explanation: behaviour over the whole population ("income is the strongest driver"). Local explanation: why *this* applicant was rejected.
• Model-specific (tree path, regression coefficient) vs model-agnostic (LIME, SHAP, PDP - work on any predict function).
• Intrinsically interpretable (white-box): linear and logistic regression (signed coefficients, odds ratios), a shallow decision tree (readable if-then path), rule lists / RuleFit, GAMs and EBMs (one smooth shape per feature, no opaque interactions), Naive Bayes (log-likelihood contributions), k-NN (explain by nearest neighbours).
• Opaque (black-box): random forests, boosted ensembles, kernel SVMs, deep networks - need post-hoc explanation.
• Accuracy-interpretability trade-off: real but not absolute; on tabular data a well-tuned GAM or a 15-feature scorecard often comes within a point or two of a boosted model's AUC, which is why banks still ship scorecards.
• Faithfulness vs plausibility: an explanation that reads well is not automatically true to the model. Sanity-check by perturbing the named features and watching the prediction move.

Post-hoc explanation techniques

• Permutation importance: shuffle one column in the *validation* set, re-score, and record the drop in performance. Model-agnostic, measures what the model actually uses on unseen data; splits credit arbitrarily between correlated features and can mislead if they are strongly correlated.
• Drop-column importance: refit without the feature - more faithful, p times costlier.
• LIME (Local Interpretable Model-agnostic Explanations): perturb one instance, get the black box's predictions, fit a weighted sparse linear model locally, read its coefficients. Fast and intuitive; unstable - two runs can differ because perturbations are random.
• SHAP (SHapley Additive exPlanations): from cooperative game theory - the Shapley value averages a feature's marginal contribution over all orderings of features. Unique method satisfying local accuracy (contributions plus baseline equal the prediction), missingness and consistency. TreeSHAP computes it exactly and fast for tree ensembles; KernelSHAP approximates for any model. Summary (beeswarm) plots give global views; force/waterfall plots give local ones.
• Partial Dependence Plot (PDP): average predicted response as one feature is swept across its range, marginalising the others - global shape. Misleads when features are correlated (it evaluates unrealistic combinations).
• ICE (Individual Conditional Expectation): one curve per instance; the average of ICE curves is the PDP. Reveals heterogeneity that a PDP hides.
• ALE (Accumulated Local Effects): uses local intervals, so it is safer than PDP under correlation.
• Surrogate model: fit a shallow tree to the black box's predictions; report its fidelity (R² or accuracy against the black box) - a low fidelity means the surrogate's story is not the model's.
• Counterfactual / actionable explanation: the smallest change that flips the decision ("income ₹60,000 higher, or one fewer enquiry in 6 months") - the form customers and regulators find most useful.
• Attention maps, Grad-CAM, integrated gradients for deep vision and text models.
• Anchors: high-precision if-then rules that pin a local prediction.

Indian practice, regulation and fairness

• DPDP Act, 2023 (Digital Personal Data Protection Act): purpose limitation and data minimisation - you may collect and process only what the stated purpose needs, which makes dropping unused features a compliance act, not just tuning. Rules under the Act have been notified in stages; check the current position before relying on specific timelines.
• RBI expects regulated lenders to be able to explain credit decisions and to govern models; its Digital Lending directions require that the borrower be told the reason for rejection, and FREE-AI / model-risk guidance pushes documented, auditable models. Quote the principle rather than a clause number unless certain of the current circular.
• Sensitive and proxy features: caste, religion, gender and in most contexts pin code must not drive lending outcomes; a dropped protected attribute can still leak through a correlated proxy, so fairness is tested on outcomes (disparate impact), not just on the feature list.
• Credit bureaus (CIBIL, Experian, CRIF, Equifax) publish "reason codes" with scores - a production example of local explanation at scale.
• Model documentation: model cards / factsheets recording features used, excluded features and why, importance ranking, and known limitations - standard in Indian bank and insurer model validation packs; IRDAI expects similar discipline in insurance pricing models.
• Clinical and public-sector AI: ICMR's ethical guidelines for AI in biomedical research and health call for explainable, auditable systems; ABDM-linked analytics inherit the same expectation.

Key points

• Feature selection - pick a subset of original columns; keeps names and meaning, unlike PCA.
• Filter - statistic vs target, model-free, fastest, ignores interactions (chi-square, ANOVA F, mutual information, correlation).
• Wrapper - search subsets by cross-validated model score; most accurate, most expensive, risk of overfitting the selection.
• Embedded - selection inside training; Lasso, tree importance, Boruta.
• L1 zeroes coefficients, L2 only shrinks - only L1-type penalties select.
• Mutual information - detects any dependence, linear or not; zero iff independent.
• Chi-square needs counts of categorical variables, never negative or standardised values.
• VIF > 5-10 signals multicollinearity; unstable coefficients, not necessarily worse predictions.
• MDI (Gini) importance is training-based and biased to high-cardinality features; permutation importance on held-out data is more trustworthy.
• SHAP - Shapley values; additive, locally accurate, consistent; TreeSHAP is exact and fast for trees.
• LIME - local sparse linear surrogate around one instance; fast but unstable across runs.
• PDP shows the average effect; ICE shows per-instance curves; ALE handles correlated features better.
• Global vs local: population behaviour vs one prediction's reason - interviewers test this distinction.
• Selection must be inside the CV fold, otherwise test performance is optimistically biased.
• Correlated features split importance between themselves; low importance does not mean irrelevant.
• Interpretability is not causality - SHAP explains the model, not the world.
• Data minimisation under the DPDP Act, 2023 and RBI's expectation of explainable credit decisions make selection and explanation regulatory requirements in Indian finance.
• Scorecard reality - banks often accept a 1-2 point AUC loss for a model they can defend.

Formulas and examples

Example 1 - Information gain of a feature (loan default). 200 applicants, 80 default (Y=1), 120 repay. Feature "salaried/self-employed": salaried 120 applicants with 30 defaults; self-employed 80 applicants with 50 defaults.

• Base entropy: p = 80/200 = 0.4, so H(Y) = −0.4 log₂0.4 − 0.6 log₂0.6 = 0.4(1.3219) + 0.6(0.7370) = 0.5288 + 0.4422 = 0.9710 bits.
• Salaried: p = 30/120 = 0.25 → H = −0.25 log₂0.25 − 0.75 log₂0.75 = 0.5 + 0.75(0.4150) = 0.8113.
• Self-employed: p = 50/80 = 0.625 → H = −0.625 log₂0.625 − 0.375 log₂0.375 = 0.625(0.6781) + 0.375(1.4150) = 0.4238 + 0.5306 = 0.9544.
• H(Y|X) = (120/200)(0.8113) + (80/200)(0.9544) = 0.4868 + 0.3818 = 0.8686.
• IG = 0.9710 − 0.8686 = 0.1024 bits. A second feature with IG 0.31 would rank above it on a filter.

Example 2 - Chi-square filter on the same table.

• Expected: salaried-default = 120×80/200 = 48; salaried-repay = 72; self-default = 32; self-repay = 48.
• χ² = (30−48)²/48 + (90−72)²/72 + (50−32)²/32 + (30−48)²/48 = 324/48 + 324/72 + 324/32 + 324/48 = 6.75 + 4.5 + 10.125 + 6.75 = 28.125, df = 1.
• Critical value at 5% for df = 1 is 3.841, so the feature is highly significant and is retained.

Example 3 - VIF and redundancy. Regressing monthly_income on annual_income, EMI and age gives R² = 0.96; regressing age on the rest gives R² = 0.21.

• VIF(monthly_income) = 1/(1 − 0.96) = 25 → severe multicollinearity (annual income carries the same information); drop one of the pair or keep only a derived ratio.
• VIF(age) = 1/(1 − 0.21) = 1/0.79 = 1.27 → fine.

Example 4 - Permutation importance with AUC. Baseline validation AUC = 0.842. Shuffling each column (mean of 5 repeats):

• Ranking: bureau score ≫ DTI > enquiries. account_vintage adds almost nothing and pin_code_hash is noise (a negative value is within sampling error) - drop both; dropping the pin-code proxy also removes a fairness risk.
• Refit on the top three: AUC 0.838, a 0.004 loss for 40% fewer features - normally accepted.

Example 5 - SHAP additivity for one applicant. Model outputs log-odds. Base value φ₀ = −1.20. Contributions: bureau_score +0.85, DTI −0.55, enquiries_6m −0.30, vintage +0.10.

• Sum: f(x) = −1.20 + 0.85 − 0.55 − 0.30 + 0.10 = −1.10 log-odds.
• Probability = 1/(1 + e^{1.10}) = 1/(1 + 3.0042) = 0.2498 ≈ 25.0% default probability.
• Local explanation / reason codes, read off the signs: the bureau_score contribution pushed the log-odds of default up (+0.85), while debt-to-income (−0.55) and recent enquiries (−0.30) pulled them down — so in this fitted model those two are risk-reducing contributions for this applicant, not risk-raising ones. Counterfactual: cutting DTI's contribution to zero gives −0.55 log-odds → probability 1/(1+e^{0.55}) = 1/(1+1.7333) = 0.366… note this *raises* risk, because DTI's negative SHAP here reduces log-odds of default; the actionable counterfactual is instead the feature whose change lowers the score.

from sklearn.feature_selection import SelectKBest, mutual_info_classif, RFECV
from sklearn.linear_model import LogisticRegression, LassoCV
from sklearn.pipeline import Pipeline
from sklearn.inspection import permutation_importance, PartialDependenceDisplay
from sklearn.model_selection import cross_val_score, StratifiedKFold

# filter + model as ONE pipeline so selection stays inside each CV fold
pipe = Pipeline([("sel", SelectKBest(mutual_info_classif, k=15)),
                 ("clf", LogisticRegression(max_iter=1000))])
print(cross_val_score(pipe, X, y, cv=StratifiedKFold(5), scoring="roc_auc").mean())

# embedded: L1 logistic -> non-zero coefficients are the selected features
l1 = LogisticRegression(penalty="l1", solver="liblinear", C=0.1).fit(Xs, y)
kept = X.columns[(l1.coef_[0] != 0)]

# wrapper: RFE with CV-chosen subset size
rfe = RFECV(LogisticRegression(max_iter=1000), step=1, cv=5, scoring="roc_auc").fit(Xs, y)

# post-hoc: permutation importance on the VALIDATION set, then SHAP
r = permutation_importance(model, X_val, y_val, n_repeats=10, scoring="roc_auc")
# import shap; shap.TreeExplainer(model).shap_values(X_val)

Where it is used

• Banking and NBFC credit risk: 10-20 feature logistic scorecards with WOE/IV binning and VIF checks at HDFC, ICICI, Axis, Bajaj Finserv; SHAP reason codes attached to boosted challenger models for adverse-action notices under RBI's digital lending expectations.
• Payments and fraud: real-time UPI/card fraud models at Paytm, PhonePe, Razorpay and Juspay where feature count drives latency; permutation importance used to prune the online feature store.
• Healthcare analytics: gene and biomarker selection (mRMR, Boruta) in IIT/AIIMS research; explainable diagnostic support where ICMR ethics guidance requires auditable reasoning.
• Insurance pricing and claims: IRDAI-regulated pricing models use GAMs/scorecards plus PDPs to show each rating factor's shape; claims-fraud triage uses local explanations for investigators.
• Telecom and e-commerce: churn and propensity models at Jio, Airtel, Flipkart, Swiggy - importance rankings feed retention campaign design.
• Job roles: data scientist, ML engineer, model validation / model risk analyst (a large function in Indian bank and GCC teams), credit risk analyst, responsible-AI and audit roles.

Interview questions

• Q: Difference between feature selection and dimensionality reduction? A: Selection keeps a subset of the original, named features, so the model stays interpretable; reduction like PCA builds new linear combinations that compress variance but lose direct meaning.
• Q: Compare filter, wrapper and embedded methods. A: Filters score features statistically against the target (fast, model-free, miss interactions); wrappers search subsets using cross-validated model performance (accurate, expensive); embedded methods select during training, like Lasso or tree importance (a good middle ground).
• Q: Why does L1 produce sparsity and L2 not? A: The L1 constraint region is a diamond with corners on the axes, so the optimum often lands exactly on an axis (coefficient = 0); L2's circular region has no corners, so coefficients shrink but stay non-zero.
• Q: Mutual information or Pearson correlation for ranking features? A: Mutual information, if the relationship may be non-linear or non-monotonic, since correlation near zero does not imply independence; correlation is cheaper and fine for roughly linear numeric relations.
• Q: Why is feature selection outside cross-validation wrong? A: Selecting on the whole dataset lets test-fold information influence the choice, which leaks and inflates the estimated score; the selector must be refitted inside each fold, usually via a pipeline.
• Q: Random forest feature_importances_ vs permutation importance? A: The built-in one is mean impurity decrease measured on training data and is biased towards high-cardinality continuous features; permutation importance measures the performance drop on held-out data and reflects what the model actually relies on.
• Q: What are SHAP values and what property makes them attractive? A: Shapley values from game theory, averaging each feature's marginal contribution over all orderings; they are locally accurate (contributions plus the base value reconstruct the prediction) and consistent, which is why they serve as reason codes.
• Q: SHAP versus LIME? A: LIME fits a local sparse linear surrogate on perturbed samples - fast but unstable; SHAP is theoretically grounded, additive and consistent, and TreeSHAP makes it exact and fast for tree ensembles, at higher cost elsewhere.
• Q: Two features are 0.95 correlated and both show low importance. Conclusion? A: Not that they are irrelevant - permutation and SHAP split credit between correlated features, so each looks weak; test by dropping one or by grouping them and permuting the group together.
• Q: When does a PDP mislead? A: When features are correlated, since it averages predictions over feature combinations that never occur in reality; use ALE, or ICE curves to expose heterogeneity.
• Q: Can SHAP tell you what causes default? A: No - it explains the trained model's behaviour on observational data; causal claims need causal identification, so a feature can be highly attributed while being a proxy, not a cause.
• Q: An Indian lender rejects a loan via a boosted model. What must you be able to produce? A: A documented local explanation - the main contributing factors in customer-comprehensible language, ideally with a counterfactual - plus model documentation and a feature list justified under data minimisation in the DPDP Act, 2023.
• Q: How would you select features for 10,000 columns and 5,000 rows? A: Start with cheap filters (variance threshold, mutual information or univariate tests) to cut to a few hundred, then L1/Elastic Net or tree gain, and only then a wrapper like RFECV on the survivors, all inside CV folds.

Common mistakes

• Running the selector on the full dataset before splitting - the classic leak that inflates test scores; always wrap selection inside the pipeline and the CV loop.
• Applying chi-square to standardised or negative-valued features, or to a continuous target, instead of counts of categorical variables.
• Reading a random forest's feature_importances_ as truth and never checking it with permutation importance or SHAP on held-out data.
• Dropping a feature because its individual correlation with the target is near zero, when it matters only in interaction with another (XOR-type relevance).
• Confusing high VIF with poor prediction - collinearity destabilises coefficients and their interpretation, but tree ensembles may predict fine.
• Treating SHAP or LIME attributions as causal effects, or as advice to the customer, without checking they are actionable.
• Dropping a protected attribute and declaring the model fair, while a correlated proxy such as pin code or occupation reproduces the same disparity.
• Forgetting to standardise features before L1/L2 regularisation or before coefficient-based RFE, so the penalty depends on units.

Related topics

Dimensionality reduction and PCA · Overfitting, bias-variance and regularisation · Decision trees and random forests · Ensemble methods and gradient boosting · Model evaluation metrics · Machine learning workflow