Machine learning workflow

Artificial Intelligence and Data Science (B.E./B.Tech) · Machine learning · free study notes for exams, placements and interviews

In short

The machine learning workflow is the end-to-end sequence that turns a business question into a deployed, monitored model: frame the problem, collect and understand data, clean and engineer features, split data, train and tune candidate models, evaluate on held-out data, deploy, then monitor and retrain. It matters because most real failures are workflow failures (leakage, a wrong split, a drifted model) rather than algorithm failures. Universities examine it as the first unit of the ML course — CRISP-DM stages, train/validation/test design, k-fold cross-validation and the preprocessing-inside-the-pipeline rule — and GATE DA tests the quantitative parts (splits, fold counts, leakage reasoning). Every Indian campus interview (TCS Digital, Infosys, Wipro, Accenture, Mu Sigma, Fractal, ZS, Flipkart, Swiggy, Jio, HDFC, Paytm) opens with "walk me through how you would build a model end to end", so this topic is effectively the frame for the whole subject.

Explained

Standard lifecycle and the reference process models

CRISP-DM (Cross-Industry Standard Process for Data Mining), the most-examined model: 1 Business understanding → 2 Data understanding → 3 Data preparation → 4 Modelling → 5 Evaluation → 6 Deployment. It is iterative and cyclic, not a one-pass waterfall; arrows run back from evaluation to business understanding.
KDD process (older, data-centric): Selection → Preprocessing → Transformation → Data mining → Interpretation/evaluation. SEMMA (SAS): Sample, Explore, Modify, Model, Assess.
Effort split in practice: data preparation typically consumes 60–80% of project time; modelling is a small slice. Figures vary by survey, so quote them as "most of the effort", not an exact number.
Two broad settings: *offline/batch* learning (train on a fixed dataset, deploy a frozen model) and *online/incremental* learning (model updates as data streams in).

Step 1–2: Problem framing and data understanding

Translate the business goal into an ML task: churn → binary classification; demand → regression; segmentation → clustering; fraud → anomaly detection or imbalanced classification; recommendation → ranking.
Decide if ML is needed at all. If a deterministic rule or SQL query answers it, use that. ML suits problems with patterns, abundant data and tolerance for probabilistic error.
Define the target variable precisely, including the time window: "churn = no transaction in the next 30 days after the observation date". A vague target invents leakage.
Fix the evaluation metric before modelling, aligned with cost: recall for disease screening, precision for ad spend, RMSE/MAPE for demand, AUC for ranking. Also fix the baseline (majority class, last-value, or existing rule) — a model that cannot beat the baseline is not a result.
Data understanding / EDA: row and column counts, data dictionary, dtypes, summary statistics, missing-value map, class balance, duplicates, cardinality of categoricals, correlation heatmap, outlier plots, target leakage suspects (columns recorded after the target).
Data sources in Indian practice: transactional RDBMS, data warehouse / lakehouse tables, APIs, logs, UIDAI/KYC-derived fields, bureau data (CIBIL), IoT telemetry, open data (data.gov.in), surveys.

Step 3: Data preparation

Cleaning: fix dtypes, parse dates, standardise units and text case, de-duplicate, correct impossible values (age 999, negative price), reconcile category spellings ("Bangalore"/"Bengaluru").
Missing values: drop rows/columns only when loss is small; otherwise impute — mean/median (numeric), mode or an explicit "Missing" level (categorical), KNN or iterative (MICE-style) imputation, forward-fill for time series. Add a missing-indicator flag when missingness is informative. Imputation parameters must be learnt on train only.
Outliers: detect by z-score (|z| > 3), IQR rule (outside Q1 − 1.5·IQR, Q3 + 1.5·IQR), or model-based (Isolation Forest). Treat by capping/winsorising, transforming (log), or keeping them if genuine.
Encoding categoricals: one-hot (low cardinality, linear models), ordinal/label (ordered categories, tree models), target/mean encoding with smoothing and out-of-fold computation (high cardinality — leakage-prone), frequency encoding, hashing (very high cardinality), embeddings (deep models).
Scaling: standardisation z = (x − μ)/σ; min–max to [0, 1]; robust scaling by median and IQR. Required for distance- and gradient-based models (k-NN, SVM, k-means, PCA, neural nets, regularised regression); not needed for decision trees, random forests and gradient boosting.
Transformations: log/Box–Cox for skew, binning/discretisation, polynomial and interaction terms, date→cyclical features (sin/cos of month), text→TF-IDF, aggregation windows for time series.
Feature engineering is the highest-leverage step: domain-derived ratios (EMI/income), recency-frequency-monetary (RFM) features, lags and rolling means, counts per customer. (Selection of features is covered in its own topic.)
Class imbalance: class weights, random undersampling, oversampling, SMOTE (synthetic minority oversampling) — resample only the training fold, never the validation or test set.

Step 4: Splitting, training and tuning

Hold-out split: typical 70:30, 80:20, or 60:20:20 for train/validation/test. Stratify on the target for classification so class proportions are preserved. Fix random_state for reproducibility.
Roles: *train* fits parameters; *validation* selects hyperparameters and models; *test* is touched once, at the end, to estimate generalisation. Tuning on the test set turns it into a validation set and inflates the reported score.
k-fold cross-validation: split train into k folds, train on k−1, validate on 1, repeat k times, average. k = 5 or 10 is standard; LOOCV is k = n (low bias, high variance, expensive). Stratified k-fold for classification; GroupKFold when rows share an entity (same patient, same customer); TimeSeriesSplit (expanding/rolling window) for temporal data — never shuffle time series.
Nested CV: an inner loop tunes hyperparameters, an outer loop estimates performance — the unbiased way to report a score when you also tuned.
Hyperparameter search: grid search (exhaustive, cost = product of grid sizes × folds), random search (better for high-dimensional spaces), Bayesian/TPE optimisation (Optuna, Hyperopt), successive halving. Hyperparameters are set before training (k in k-NN, C in SVM, tree depth, learning rate); parameters are learnt during training (weights, split thresholds).
Model selection: try a baseline, a linear/logistic model, a tree ensemble, and (if justified) a neural model; compare with the same folds and metric; prefer the simpler model when scores tie (Occam's razor, easier to deploy and explain).
Pipelines: chain preprocessing + estimator into one object so every fold refits the preprocessing. This is the structural cure for leakage and makes the artefact deployable as a single unit.

Step 5: Evaluation and error analysis

Report on the untouched test set, with the metric fixed in step 1 plus a confusion matrix or residual plots; include a confidence interval or fold-wise standard deviation rather than a bare number.
Diagnose fit: large train–validation gap → overfitting (regularise, simplify, add data); both poor → underfitting (richer model, better features). Use learning curves (score vs training size) and validation curves (score vs one hyperparameter).
Error analysis: inspect misclassified cases, slice performance by segment (city, tenure, channel, device) — aggregate accuracy can hide failure on a subgroup.
Threshold tuning for classification: the 0.5 cut-off is arbitrary; choose it from the precision–recall curve or expected cost. Calibration (Platt scaling, isotonic) matters when probabilities are used for pricing or credit limits.
Business validation: translate the metric into money or risk, and run an A/B test or champion–challenger trial before full rollout.

Step 6: Deployment, monitoring and MLOps

Packaging: serialise the fitted pipeline (joblib/pickle, ONNX for portability) with pinned library versions; version data, code and model together (Git + DVC + MLflow Model Registry).
Serving patterns: batch scoring (nightly jobs writing to a table), real-time REST/gRPC API (FastAPI/Flask in a container), streaming, and on-device/edge inference.
Training–serving skew: the same transformation code must run in both places — a feature store or a shared pipeline object prevents mismatch.
Monitoring: operational (latency, throughput, errors), data drift (input distribution shifts — PSI, KS test, chi-square), concept drift (the X→y relationship changes), label delay, and live metric tracking once ground truth arrives. Set retraining triggers: schedule-based, drift-based, or performance-based.
Governance in India: the Digital Personal Data Protection Act, 2023 governs personal-data processing (notice, consent, purpose limitation, data-minimisation, Data Principal rights, Significant Data Fiduciary duties) — its rules have been notified in phases with staged compliance timelines, so check the current stage before quoting dates. The RBI requires explainability, audit trails and board-approved model governance for credit and fraud models (plus its digital-lending guidelines), SEBI and IRDAI have analogous expectations, and IndiaAI/MeitY frameworks and the ISO/IEC 42001 AI management-system standard are increasingly cited in enterprise reviews. Treat sector-specific thresholds as subject to change.
Documentation and reproducibility: data dictionary, model card (intent, data, metrics, limitations), experiment logs, seeds, environment file.

Key points

ML workflow - the repeatable loop from business question to monitored production model, not a one-time model fit.
CRISP-DM - business understanding, data understanding, data preparation, modelling, evaluation, deployment; iterative.
Target definition - the label plus its time window; a loose definition is the commonest source of leakage.
Baseline first - majority class, mean, or existing rule; a model is only good relative to it.
Metric chosen upfront - recall, precision, F1, AUC, RMSE, MAPE picked from business cost, not convenience.
Train / validation / test - fit, select, and estimate respectively; the test set is used exactly once.
Stratified split - preserves class proportions, essential for imbalanced classification.
k-fold CV - averages k train/validate rounds (k = 5 or 10) for a lower-variance estimate; LOOCV is k = n.
TimeSeriesSplit - train on past, validate on future; random shuffling of temporal data leaks the future.
Data leakage - any information in training that will not exist at prediction time; symptom is a suspiciously perfect score.
Fit on train, transform on test - scalers, imputers and encoders learn only from training data.
Pipeline - preprocessing + model as one object; refits per fold and ships as one deployable artefact.
Hyperparameters vs parameters - set before training vs learnt during training.
Search methods - grid (exhaustive), random (efficient in high dimensions), Bayesian (model-guided).
Scaling need - mandatory for k-NN, SVM, k-means, PCA, neural nets; irrelevant for tree-based models.
Drift - data drift (inputs shift) and concept drift (relationship shifts) both demand retraining.
Training–serving skew - different transformation code in training and inference silently degrades live accuracy.
DPDP Act, 2023 - India's personal-data law: consent, purpose limitation, minimisation; phased rule notification.

Formulas and examples

Split sizes: n_train = ⌈(1 − p_test) × n⌉, n_test = ⌊p_test × n⌋ — n = total rows, p_test = test fraction (dimensionless)
k-fold: fold size ≈ n/k; models trained = k; rows used for training per fold = n × (k − 1)/k
Grid-search fits = (∏ of grid sizes) × k (+1 final refit on full training data)
Standardisation: z = (x − μ_train)/σ_train — μ, σ in the feature's own units; z is dimensionless
Min–max: x' = (x − x_min)/(x_max − x_min) ∈ [0, 1]
IQR outlier bounds: [Q1 − 1.5 × IQR, Q3 + 1.5 × IQR], IQR = Q3 − Q1 (feature units)
CV estimate: mean = (1/k) Σ sᵢ ; s.d. = √[Σ(sᵢ − mean)² / (k − 1)] — sᵢ = score of fold i
Population Stability Index: PSI = Σ (a_i − e_i) × ln(a_i / e_i) — a_i, e_i = actual and expected proportions in bin i
SMOTE synthetic point: x_new = x_i + λ (x_nn − x_i), λ ~ U(0, 1) — x_nn is a minority-class neighbour

Example 1 — Split and fold arithmetic. A telecom churn dataset has 50,000 rows with 8% churners. Plan: 20% test, 5-fold CV on the remainder.

Test = 0.20 × 50,000 = 10,000 rows; training pool = 40,000 rows.
Stratification keeps 8% churn in both: test churners = 0.08 × 10,000 = 800; training churners = 0.08 × 40,000 = 3,200.
5-fold on 40,000 → fold size = 40,000/5 = 8,000; each model trains on 32,000 and validates on 8,000, with about 8,000 × 0.08 = 640 churners per validation fold.
Models fitted during CV = 5.

Example 2 — Grid-search cost and runtime. Tuning a random forest: n_estimators ∈ {100, 300, 500}, max_depth ∈ {5, 10, 20, None}, min_samples_leaf ∈ {1, 5}, with 5-fold CV.

Combinations = 3 × 4 × 2 = 24; fits = 24 × 5 = 120, plus 1 refit = 121.
At 12 s per fit on one core: 121 × 12 = 1,452 s ≈ 24.2 minutes. With n_jobs = 4 (ideal scaling) ≈ 6.1 minutes.
Random search with n_iter = 10 → 10 × 5 + 1 = 51 fits ≈ 10.2 min single-core — about 58% less work than the grid.

Example 3 — Correct vs leaky scaling. Train ages: 20, 30, 40, 50, 60 → μ = 200/5 = 40, σ (population) = √[(400+100+0+100+400)/5] = √200 ≈ 14.14.

Test row age = 70 → correct z = (70 − 40)/14.14 ≈ +2.12 (flagged as unusually old).
Leaky version: fit the scaler on train+test together → μ = (200 + 70)/6 = 45, σ = √[(625+225+25+25+225+625)/6] = √(1750/6) ≈ 17.08 → z = (70 − 45)/17.08 ≈ +1.46. The test point has pulled its own statistics, understating the shift and inflating the validation score.

Example 4 — CV mean, spread and model choice. Accuracy over 5 folds — Model A: 0.84, 0.86, 0.85, 0.83, 0.87; Model B: 0.90, 0.78, 0.88, 0.76, 0.93.

A: sum = 4.25 → mean = 0.850; deviations −0.01, +0.01, 0, −0.02, +0.02 → Σd² = 0.0001+0.0001+0+0.0004+0.0004 = 0.0010 → s.d. = √(0.0010/4) = 0.0158.
B: sum = 4.25 → mean = 0.850; deviations +0.05, −0.07, +0.03, −0.09, +0.08 → Σd² = 0.0025+0.0049+0.0009+0.0081+0.0064 = 0.0228 → s.d. = √(0.0228/4) = 0.0755.
Same mean, but B's spread is ~4.8× larger → choose A: more stable across data slices, and its single reported number is more trustworthy.

Example 5 — Drift check with PSI. Expected (training) vs actual (last month) distribution of a credit-score band.

Band	Expected eᵢ	Actual aᵢ	aᵢ − eᵢ	ln(aᵢ/eᵢ)	Contribution
Low	0.20	0.30	0.10	0.4055	0.0405
Medium	0.50	0.45	−0.05	−0.1054	0.0053
High	0.30	0.25	−0.05	−0.1823	0.0091

PSI = 0.0405 + 0.0053 + 0.0091 ≈ 0.055. Conventional reading: < 0.10 no significant shift, 0.10–0.25 moderate (investigate), > 0.25 major (retrain). So: monitor, no retrain yet.

Stage	Main risk	Control
Problem framing	Wrong target or metric	Written target definition, cost-based metric, baseline
Data preparation	Leakage, lost rows	Fit transforms on train only; impute with indicators
Splitting	Shuffled time series, grouped rows split	TimeSeriesSplit, GroupKFold, stratification
Tuning	Overfitting the validation set	Nested CV, fewer peeks, simpler grids
Evaluation	Optimistic score	Untouched test set, fold s.d., slice analysis
Deployment	Training–serving skew	Ship the pipeline object, shared feature code
Post-deployment	Silent decay	Drift metrics (PSI/KS), retraining triggers

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, GridSearchCV, train_test_split

X_tr, X_te, y_tr, y_te = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42)      # test set sealed here

num = Pipeline([("imp", SimpleImputer(strategy="median")),
                ("sc", StandardScaler())])
cat = Pipeline([("imp", SimpleImputer(strategy="most_frequent")),
                ("oh", OneHotEncoder(handle_unknown="ignore"))])

pipe = Pipeline([
    ("prep", ColumnTransformer([("n", num, num_cols), ("c", cat, cat_cols)])),
    ("clf", LogisticRegression(max_iter=1000, class_weight="balanced"))])

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
gs = GridSearchCV(pipe, {"clf__C": [0.01, 0.1, 1, 10]},
                  scoring="roc_auc", cv=cv, n_jobs=-1)
gs.fit(X_tr, y_tr)                 # preprocessing refits inside every fold
print(gs.best_params_, gs.best_score_)
print("test AUC:", gs.score(X_te, y_te))   # touched once

Where it is used

Banking, NBFCs and fintech: credit scoring, collections prioritisation and fraud models at HDFC, ICICI, Axis, Bajaj Finserv, Paytm, PhonePe and CRED — built under RBI model-governance expectations with audit trails and periodic PSI-based revalidation.
E-commerce, food delivery and mobility: demand forecasting, ETA prediction, recommendation and pricing at Flipkart, Amazon India, Myntra, Swiggy, Zomato, Zepto, Ola and Uber, usually with batch training plus real-time serving and A/B tests.
IT services and analytics firms: TCS, Infosys, Wipro, Accenture, Cognizant, Mu Sigma, Fractal, ZS Associates and Quantiphi deliver client projects structured explicitly as CRISP-DM phases; the workflow is the client deliverable.
Telecom, insurance and healthtech: churn and network-fault prediction at Jio and Airtel, claims triage at insurers (IRDAI-regulated), and diagnostics support at Apollo and Practo-type platforms, where recall and calibration dominate.
Public sector and agritech: data.gov.in-based dashboards, crop-yield and pest-risk models with ICAR and ISRO imagery, and welfare targeting — governed by DPDP Act obligations on personal data.
Job roles: data scientist, ML engineer, data analyst, MLOps/platform engineer, decision scientist; interviews and performance reviews centre on pipeline hygiene, reproducibility and monitoring, not on inventing algorithms.

Interview questions

Q: Walk me through an end-to-end ML project. A: Frame the problem and metric with a baseline, explore and clean the data, engineer features, split train/validation/test, train and tune candidates with cross-validation inside a pipeline, evaluate once on the test set, deploy the fitted pipeline, then monitor drift and retrain.
Q: What is data leakage and how do you prevent it? A: Leakage is using information at training time that will not be available at prediction time (future columns, target-derived features, or statistics computed over the full dataset). Prevent it by sealing the test set first, fitting all transforms inside a pipeline on training folds only, and auditing each feature for when it becomes known.
Q: Why have a validation set when you already have a test set? A: Validation data is used repeatedly to pick models and hyperparameters, which biases its score upward; the test set stays untouched to give an honest estimate of generalisation.
Q: When would you not use k-fold cross-validation? A: With temporal data (use TimeSeriesSplit), with grouped rows like multiple records per customer (use GroupKFold), or when training is so expensive that a single large hold-out is the only practical option.
Q: Why must SMOTE be applied only to the training fold? A: Synthesising minority points using validation rows leaks their information into training and duplicates near-identical points across the split, so the validation score becomes optimistic and meaningless.
Q: Difference between a parameter and a hyperparameter? A: Parameters are learnt from data during fitting (regression weights, tree split thresholds); hyperparameters are configured before fitting (C, k, learning rate, max_depth) and are chosen by validation.
Q: Which algorithms need feature scaling and which do not? A: Distance- and gradient-based methods need it — k-NN, SVM, k-means, PCA, neural networks, regularised regression; decision trees, random forests and gradient boosting are invariant to monotonic rescaling.
Q: Your model scores 0.99 AUC on the first run. What do you check? A: Suspect leakage: look for a column that encodes the target or is recorded after it, a duplicated row shared across splits, or preprocessing fitted on the whole dataset before splitting.
Q: How do you decide the classification threshold? A: Not by defaulting to 0.5 — pick it on validation data from the precision–recall curve or by minimising expected cost of false positives versus false negatives, then verify on the test set.
Q: Model accuracy was 88% at launch and 79% six months later. Diagnose it. A: Check input data drift (PSI/KS per feature) and pipeline or upstream schema changes first, then concept drift where the relationship itself has shifted; retrain on recent data and, if drift is structural, re-engineer features.
Q: What is training–serving skew? A: When the transformations applied at inference differ from those used in training — different code, defaults or feature freshness — so the live model sees inputs it was never trained on; shipping the fitted pipeline or using a feature store avoids it.
Q: How do you make an ML project reproducible? A: Version data, code and model artefacts (Git, DVC, MLflow), pin library versions and random seeds, log every experiment's configuration and metric, and keep preprocessing in code rather than in manual spreadsheet steps.
Q: What Indian regulation affects your workflow? A: The DPDP Act, 2023 requires notice, consent, purpose limitation and data minimisation for personal data (from 13 May 2027 under the DPDP Rules 2025; until then IT Act s.43A and the SPDI Rules 2011 govern), and sectoral regulators like RBI and IRDAI additionally expect explainability, documentation and periodic model validation.

Common mistakes

Fitting scalers, imputers or encoders on the whole dataset before splitting — the most common leakage error in labs and take-home assignments.
Tuning hyperparameters against the test set, then reporting that same score as the generalisation estimate.
Random shuffling of time-series data, or splitting rows that belong to the same customer or patient across train and test.
Oversampling or SMOTE-ing before the split, which copies minority points into both sides and produces near-perfect fake scores.
Reporting accuracy on a 98:2 imbalanced dataset — 98% accuracy equals the do-nothing baseline; use recall, precision, F1 or AUC.
Skipping EDA and the baseline, jumping straight to a complex model, and having nothing to compare against or explain.
Treating deployment as the finish line — no monitoring, no drift check, no retraining trigger, so the model decays unnoticed.
Quoting a single CV mean without its fold-wise spread, hiding a model that is unstable across data slices.