Process statistics and data analytics

In short

Process statistics is the use of statistical methods to turn plant and laboratory measurements into reliable numbers: averages with uncertainty, calibration equations, tests of whether a change really improved anything, and charts that say whether a process is in control. It matters because every flow, temperature, composition and yield used in a material or energy balance is a *measurement* carrying error, and decisions worth crores are taken on those numbers. It is examined in GATE (CH) under probability, statistics, regression and error analysis, and in university papers on process calculations, instrumentation and quality control. Indian employers — Reliance, IOCL, BPCL, Tata Chemicals, UPL, Aarti Industries, Deepak Nitrite, Asian Paints, Pfizer/Sun Pharma, and the analytics arms of TCS, L&T Technology Services, Honeywell and Yokogawa — test it in technical rounds for process, quality, QA/QC, RTO and digital-twin roles.

Explained

Data types, sampling and descriptive statistics

• Population vs sample: a population is every possible measurement; a sample is the few you actually took. Statistics estimate population parameters (μ, σ) from sample values (x̄, s).
• Data types: continuous (temperature, flow), discrete/count (number of defective bags), attribute (pass/fail), categorical (grade A/B/C). The data type decides which chart and test are valid.
• Measures of central tendency: mean (sensitive to outliers), median (robust — use for skewed data such as particle size or downtime), mode (most frequent, used for attribute data).
• Measures of dispersion: range, variance s², standard deviation s, interquartile range (Q3 − Q1), coefficient of variation CV = s/x̄ (dimensionless, used to compare instruments of different spans).
• Shape: skewness (asymmetry; residence-time and particle-size data are right-skewed), kurtosis (peakedness/tail heaviness).
• Why n − 1: dividing by degrees of freedom (n − 1) makes s² an unbiased estimator because the mean itself was estimated from the same data.
• Graphical tools: histogram, box plot, run chart (time order — never destroy time order), scatter plot, Pareto chart (80/20 of defect causes), cause-and-effect (Ishikawa) diagram.
• Sampling plans: random, stratified (by shift, reactor, batch), systematic (every 10th bag), cluster. Autocorrelated process data (DCS tags every second) are *not* independent — averaging them underestimates true variability.

Probability distributions used in process work

• Normal (Gaussian): symmetric, defined by μ and σ; random measurement error, and sums/averages of many small errors. 68.27% within ±1σ, 95.45% within ±2σ, 99.73% within ±3σ (the basis of 3σ control limits).
• Standard normal z = (x − μ)/σ; used for fraction-out-of-spec calculations.
• Student's t: like normal but fatter tails; used when σ is unknown and estimated by s, especially for small n (< 30).
• Chi-square (χ²): distribution of sample variance; used for tests on variance and goodness-of-fit.
• F distribution: ratio of two variances; the test statistic in ANOVA and in comparing instrument repeatability.
• Binomial: number of defectives in n independent trials with constant probability p — attribute inspection.
• Poisson: number of rare events per unit (pinholes per m² of film, leaks per km of pipeline, trips per year); mean = variance = λ.
• Log-normal: variable whose logarithm is normal — particle size, droplet size, pollutant concentrations, equipment life.
• Weibull: failure time of equipment; shape parameter β < 1 infant mortality, β = 1 random, β > 1 wear-out. Basis of reliability and maintenance planning.
• Exponential: time between random failures or between arrivals; memoryless.
• Central Limit Theorem: the distribution of the sample mean tends to normal with standard error σ/√n whatever the parent distribution — this is why subgroup averages are charted rather than individual readings.

Measurement error, uncertainty and propagation

• Accuracy vs precision: accuracy is closeness to the true value (freedom from bias); precision is repeatability (small random scatter). An instrument can be precise and badly biased.
• Error classes: (i) random error — scatter, reduced by averaging; (ii) systematic error/bias — calibration offset, drift, unremoved zero, reduced only by calibration; (iii) gross error — leak, blocked impulse line, wrong units, transcription mistake, removed only by detection and correction.
• Absolute vs relative error: ±0.5 °C vs ±0.2%. Instrument specs are often "% of full scale", so error at low flow is a large *relative* error — a key interview point for orifice meters.
• Propagation of error (independent variables): combine variances, not errors; for a product or quotient, relative variances add.
• Standard uncertainty, combined uncertainty, expanded uncertainty: U = k·u_c, coverage factor k = 2 for roughly 95% confidence. NABL-accredited calibration laboratories in India must report the expanded uncertainty with the calibration certificate; the ISO/BIPM Guide to the Expression of Uncertainty in Measurement (GUM) is the method, adopted by BIS (quote the subject rather than an IS number unless verified).
• Significant figures: a computed result cannot be more precise than its least precise input — a recurring exam penalty.
• Rounding rule: carry full precision through the calculation, round only the final answer.

Regression, correlation and curve fitting

• Least squares principle: choose parameters that minimise Σ(residual)² — residual = observed − predicted.
• Simple linear regression y = a + bx: used for instrument calibration (mV vs temperature, absorbance vs concentration in a Beer–Lambert curve, ΔP vs flow²).
• Coefficient of determination R²: fraction of variance in y explained by the model; R² always rises when terms are added, so use adjusted R² for model comparison.
• Correlation coefficient r: −1 to +1, strength of *linear* association only. Correlation is not causation — in plant data, feed rate and cooling-water flow correlate because both follow production rate.
• Residual analysis: plot residuals against x and against time; curvature means a wrong model form, a funnel shape means non-constant variance (heteroscedasticity), drift means a missing time-dependent variable. R² alone never validates a model.
• Linearising nonlinear models: Arrhenius k = A exp(−E/RT) → ln k vs 1/T; power law y = ax^b → log–log; this is how activation energy and reaction order are extracted in kinetics experiments.
• Multiple linear regression and multicollinearity: correlated inputs make coefficients unstable; detected by the variance inflation factor, cured by PCA/PLS or by dropping variables.
• Interpolation vs extrapolation: a calibration is valid only inside the fitted range.
• Weighted least squares: weight each point by 1/σ² when measurement precision varies — the correct treatment for data spanning decades of concentration.

Hypothesis testing, ANOVA and design of experiments

• Steps: state H₀ (no effect) and H₁; choose significance level α (usually 0.05); compute the test statistic; compare with the critical value or use the p-value; conclude in process language.
• p-value: probability of seeing data at least this extreme if H₀ were true. p < α → reject H₀. It is *not* the probability that H₀ is true.
• Type I error (α): rejecting a true H₀ — stopping a good plant. Type II error (β): accepting a false H₀ — missing a real problem. Power = 1 − β, raised by larger n or a bigger real effect.
• Common tests: one-sample t (mean vs a target), two-sample t (two catalysts, two shifts), paired t (before/after on the same units), F test (two variances), χ² (goodness-of-fit, contingency), z test (large n, known σ).
• ANOVA: splits total variation into between-treatment and within-treatment (error); F = MS_between/MS_within. One-way (one factor, e.g. four catalyst batches), two-way (factor plus block).
• Design of experiments (DOE): vary factors simultaneously in a planned pattern instead of one-factor-at-a-time, which misses interactions and needs more runs.
  • Full factorial 2^k: k factors at two levels; gives all main effects and interactions. 2³ = 8 runs.
  • Fractional factorial 2^k−p: screening many factors cheaply; effects are *confounded* (aliased).
  • Response surface methodology (RSM): central composite or Box–Behnken designs fit a quadratic model to locate an optimum (e.g. maximise conversion over temperature, pressure, catalyst loading).
  • Randomisation, replication, blocking — the three principles: randomisation removes bias from unknown drifts, replication estimates error, blocking removes a known nuisance factor (raw-material lot, day).
  • Taguchi methods: orthogonal arrays and the signal-to-noise ratio; robust design against noise factors. Widely used in Indian manufacturing quality circles.

Statistical process control, capability and data reconciliation

• Common-cause vs special-cause variation: common cause is inherent noise (leave the process alone); special cause is an assignable event (investigate). Tampering with a stable process increases variance — Deming's funnel lesson.
• Shewhart control charts: centre line and ±3σ limits computed *from the process*, never from the specification.
  • Variables: X̄–R (subgroups up to about 9), X̄–s (larger subgroups), individuals–moving range (I–MR) for one reading per batch.
  • Attributes: p (fraction defective), np (number defective), c (defects per unit, constant area), u (defects per unit, varying area).
  • CUSUM and EWMA: detect small sustained shifts much faster than Shewhart charts; used on analyser and emission data.
• Western Electric run rules: 1 point beyond 3σ; 2 of 3 beyond 2σ; 4 of 5 beyond 1σ; 8 consecutive on one side; trends of 7; cycling.
• Indian standards: IS 397 covers methods of statistical quality control during production — control charts for variables, control charts for attributes and cumulative sum charts in its parts. IS 2500 covers sampling inspection procedures (attributes and variables), harmonised with ISO 2859 and ISO 3951. BIS product licences and Hallmarking-type schemes rely on these sampling plans.
• Process capability: Cp compares spec width to process spread; Cpk also penalises off-centring. Cp ≥ 1.33 is the usual industry minimum, Cp ≥ 1.67 for safety-critical and pharma. Pp/Ppk use long-term (overall) variation; a capability index is meaningless unless the chart first shows statistical control.
• Six Sigma and DMAIC: Define–Measure–Analyse–Improve–Control; 6σ capability ≈ 3.4 defects per million opportunities with the standard 1.5σ long-term shift allowance. Measurement System Analysis (gauge R&R) must show the gauge consumes < 10% of tolerance before any plant data is trusted.
• Data reconciliation: plant measurements never satisfy the mass balance exactly; reconciliation adjusts them by weighted least squares, weights 1/σ², subject to the balance constraints — producing one consistent set of flows. Redundancy (more measurements than needed) makes it possible.
• Gross error detection: test each constraint residual (or measurement adjustment) against its standard deviation; |z| > about 1.96–3 flags a faulty instrument or a leak. Reconciling data that contains a gross error smears the error over every good meter — detect first, then reconcile.
• Multivariate and machine-learning methods: PCA compresses hundreds of correlated DCS tags into a few principal components, with Hotelling's T² and SPE (Q) charts for multivariate SPC; PLS regression and PCR relate process tags to quality; soft sensors (inferential models) predict a lab property such as distillate purity between hourly samples; classification trees, random forests and neural networks for fault diagnosis and predictive maintenance. All need scaling, outlier removal, train/validation/test splits and guarding against overfitting; a model that extrapolates outside its training envelope must not be trusted on a live plant.
• Indian regulatory data practice: CPCB-mandated online continuous emission and effluent monitoring (OCEMS) in the 17 categories of highly polluting industries produces statistically audited time series; BEE's PAT scheme under the Energy Conservation Act requires measurement and verification of specific energy consumption with stated uncertainty; pharmaceutical plants follow Schedule M (Drugs and Cosmetics Rules) and ICH Q8–Q10, where annual product quality review uses trend and capability analysis, and 21 CFR Part 11-style data integrity (ALCOA+) rules govern electronic records.

Key points

• Population vs sample — parameters μ, σ are unknown; x̄, s estimate them; use n − 1 for s².
• Standard error — σ/√n; the average of 9 readings is 3 times more precise than one reading, not 9.
• Central Limit Theorem — sample means are near-normal regardless of the parent distribution; the basis of control charts and confidence intervals.
• Confidence interval — x̄ ± t·s/√n; a 95% CI is about the *mean*, not about individual future readings.
• Accuracy vs precision — bias vs scatter; averaging kills scatter but never bias.
• Error propagation — variances add; for products and quotients, *relative* variances add; the largest relative error dominates.
• Least squares — minimises Σ(residuals)²; always check residual plots, never R² alone.
• Correlation ≠ causation — plant tags correlate because both follow throughput.
• Arrhenius linearisation — plot ln k vs 1/T; slope = −E/R.
• p-value — probability of the data under H₀, not the probability that H₀ is true.
• Type I / Type II — false alarm vs missed detection; power = 1 − β rises with n.
• DOE beats OFAT — factorial designs reveal interactions and need fewer runs; randomise, replicate, block.
• Common vs special cause — adjusting a stable process (tampering) makes it worse.
• Control limits ≠ specification limits — limits come from the process, specs from the customer.
• Cp vs Cpk — Cp is potential (spread only), Cpk is actual (spread + centring); Cpk ≤ Cp always.
• Capability needs control first — an out-of-control process has no single σ to quote.
• Data reconciliation — weighted least squares forces measurements to obey mass balances; requires redundancy.
• Gross errors first — detect and remove a faulty meter before reconciling, or its error contaminates every flow.
• Indian codes — IS 397 (SQC control charts), IS 2500 (sampling inspection), GUM/NABL for measurement uncertainty, CPCB OCEMS for emission data.

Formulas and examples

Example 1 — Descriptive statistics, confidence interval and a t test. Eight flow readings (m³/h): 50.2, 49.8, 50.5, 50.1, 49.6, 50.3, 50.0, 49.9.

1. Σx = 400.4 → x̄ = 400.4/8 = 50.05 m³/h.
2. Σ(x − x̄)² = 0.0225 + 0.0625 + 0.2025 + 0.0025 + 0.2025 + 0.0625 + 0.0025 + 0.0225 = 0.5800.
3. s² = 0.5800/7 = 0.08286 → s = 0.288 m³/h; CV = 0.288/50.05 = 0.58%.
4. SE = 0.288/√8 = 0.288/2.828 = 0.1018. With t(0.025, 7) = 2.365, 95% CI = 50.05 ± 0.241 = (49.81, 50.29) m³/h.
5. Is the mean different from the design value 50.5? t = (50.05 − 50.5)/0.1018 = −4.42; |t| > 2.365, so reject H₀ — the meter reads significantly below design (and 50.5 lies outside the CI, the same conclusion).

Example 2 — Propagation of error in a mass flow. ṁ = ρQ, with ρ = 998 ± 2 kg/m³ and Q = 0.0120 ± 0.0003 m³/s (standard uncertainties).

1. ṁ = 998 × 0.0120 = 11.976 kg/s.
2. Relative uncertainties: ρ → 2/998 = 0.00200 (0.20%); Q → 0.0003/0.0120 = 0.0250 (2.50%).
3. Combined: √(0.00200² + 0.0250²) = √(4.02×10⁻⁶ + 6.25×10⁻⁴) = √(6.290×10⁻⁴) = 0.0251 = 2.51%.
4. u_ṁ = 11.976 × 0.0251 = 0.30 kg/s; expanded U (k = 2) = ±0.60 kg/s → report 11.98 ± 0.60 kg/s (95%).
5. Insight: flow contributes (2.50/2.51)² ≈ 99% of the variance — improving the density measurement is pointless; recalibrate the flowmeter.

Example 3 — Calibration by least squares. Analyser output y (mA) against known concentration x (%): (1, 2.1), (2, 4.0), (3, 6.1), (4, 8.1), (5, 9.9).

1. x̄ = 3.0, ȳ = 30.2/5 = 6.04.
2. Σ(x−x̄)(y−ȳ) = 7.88 + 2.04 + 0 + 2.06 + 7.72 = 19.70; Σ(x−x̄)² = 10.
3. b = 19.70/10 = 1.97 mA per %; a = 6.04 − 1.97(3) = 0.13 mA. Equation: y = 0.13 + 1.97x.
4. Residuals: 0.00, −0.07, +0.06, +0.09, −0.08 → SS_res = 0.0230; SS_tot = 38.832 → R² = 1 − 0.0230/38.832 = 0.9994.
5. s_e = √(0.0230/3) = 0.088 mA; SE(b) = 0.088/√10 = 0.028, so b = 1.97 ± 0.09 (95%, t = 3.182). Inverse use: x = (y − 0.13)/1.97, valid only for 1–5%.

Example 4 — X̄–R chart and capability. 20 subgroups of n = 5 reactor-outlet purity readings give X̄̄ = 50.00%, R̄ = 2.30%. Specification 50 ± 3%. For n = 5: A₂ = 0.577, D₄ = 2.114, D₃ = 0, d₂ = 2.326.

1. X̄ chart: UCL = 50.00 + 0.577 × 2.30 = 51.33, LCL = 48.67.
2. R chart: UCL = 2.114 × 2.30 = 4.86, LCL = 0.
3. σ̂ = R̄/d₂ = 2.30/2.326 = 0.989%.
4. Cp = (53 − 47)/(6 × 0.989) = 6/5.934 = 1.01 — below the 1.33 minimum, so the spread itself is too wide.
5. If the mean drifts to 50.8%: Cpk = min[(53 − 50.8), (50.8 − 47)]/(3 × 0.989) = min[2.2, 3.8]/2.967 = 0.74 — centring loss alone raises the expected out-of-spec fraction about fivefold (≈0.24% centred to ≈1.3%).

Example 5 — Data reconciliation with a gross-error check. One pipe, two meters: FT-1 reads 100.0 t/h (σ = 2.0), FT-2 reads 103.0 t/h (σ = 1.0).

1. Imbalance r = 100.0 − 103.0 = −3.0; σ_r = √(2.0² + 1.0²) = √5 = 2.236.
2. z = −3.0/2.236 = −1.34; |z| < 1.96 → no gross error at 95%, so reconcile.
3. Weights 1/σ²: 0.25 and 1.00. x̂ = (0.25 × 100.0 + 1.00 × 103.0)/1.25 = 128.0/1.25 = 102.4 t/h.
4. σ of the estimate = 1/√1.25 = 0.894 t/h — better than either meter alone, the benefit of redundancy.
5. Adjustments: FT-1 +2.4 (1.2σ), FT-2 −0.6 (0.6σ) — the less precise meter is moved more, as it should be.

Where it is used

• Refineries and petrochemicals (IOCL, BPCL, HPCL, RIL Jamnagar, GAIL): daily oil-accounting and yield reconciliation, loss-control reports, soft sensors for distillation product properties, advanced process control benefit monitoring.
• Fertiliser, chemicals and specialty plants (IFFCO, RCF, Tata Chemicals, UPL, Aarti, Deepak Nitrite, SRF): SPC on product purity and moisture, DOE for yield and catalyst optimisation, energy-intensity benchmarking under BEE's PAT scheme.
• Pharmaceuticals and biotech (Sun Pharma, Cipla, Dr Reddy's, Biocon, Serum Institute): Quality by Design (ICH Q8), DOE for critical process parameters, annual product quality review trending, stability-data regression for shelf life, Schedule M compliance.
• FMCG, paints and cement (HUL, Nestlé India, Asian Paints, UltraTech, ACC): sampling inspection to IS 2500, gauge R&R, Six Sigma Green/Black Belt projects, Pareto-driven defect reduction.
• Environment and safety: CPCB OCEMS stack and effluent data validation, statistical compliance against consent limits, reliability and Weibull analysis for relief valves and rotating equipment.
• Job roles: process engineer, quality assurance/QC engineer, production planning and yield accounting, process data scientist and digital-twin engineer (Honeywell, Yokogawa, AspenTech, Siemens, TCS, L&T Technology Services, Accenture), Six Sigma specialist; typical toolkits are Excel, Minitab, JMP, MATLAB and Python (NumPy, pandas, scikit-learn, statsmodels).

Interview questions

• Q: Why divide by n − 1 instead of n for sample variance? A: One degree of freedom is consumed in estimating the mean from the same data; dividing by n − 1 makes the variance estimate unbiased.
• Q: Difference between accuracy and precision, with a plant example? A: A flowmeter repeating 95.0 ± 0.1 t/h when the true flow is 100 t/h is precise but inaccurate — a calibration bias that averaging will never remove.
• Q: Two meters on one line read 100 and 103 t/h. Which do you believe? A: Neither outright — test the imbalance against the combined uncertainty for a gross error, then reconcile by weighting each reading by 1/σ², which here gives about 102.4 t/h.
• Q: Can control limits be set from the specification? A: No. Control limits describe what the process actually does (±3σ); using specs makes the chart blind to real shifts and triggers false alarms.
• Q: Process is in control but Cpk is 0.8 — what do you do? A: Stable but incapable: reduce common-cause variation or re-centre the mean, through process or design change, not by adjusting the operation run to run.
• Q: Why is a factorial design better than changing one variable at a time? A: It estimates interactions, gives each effect from all runs (so better precision per run), and covers the design space in fewer experiments.
• Q: What does a p-value of 0.03 mean? A: If the null hypothesis were true, data this extreme would occur 3% of the time; at α = 0.05 you reject H₀, but it is not a 3% chance that H₀ is true.
• Q: An orifice meter is quoted ±1% of full scale. Why is that a problem at low throughput? A: The error is fixed in absolute terms, so at 20% of range it is ±5% of the reading — the reason turndown ratio matters in meter selection.
• Q: How would you build a soft sensor for distillate purity? A: Collect synchronised DCS tags with lab reference values, remove outliers and align time delays, fit a PLS or regression model on scaled data, validate on held-out data, then monitor residuals and retrain on drift.
• Q: Which Indian standards govern statistical quality control and sampling? A: IS 397 for control charts for variables, attributes and cumulative sums during production, and IS 2500 for sampling inspection procedures, harmonised with ISO 2859 and ISO 3951.
• Q: Why can a high R² still mean a bad model? A: R² only measures variance explained; curvature, heteroscedasticity, autocorrelation or a single influential outlier can give R² above 0.99 with a model that predicts badly — residual plots expose it.
• Q: What is tampering, and why is it harmful? A: Reacting to common-cause variation as if it were a special cause; the corrective moves add their own variance and inflate overall variability.

Common mistakes

• Treating control limits as specification limits, or computing Cp/Cpk before the chart shows statistical control.
• Adding errors instead of variances in propagation, or ignoring that relative errors combine in products and quotients.
• Reporting more significant figures than the data support, or rounding intermediate steps and then quoting four decimals.
• Using the z value instead of t for a small sample, or forgetting that degrees of freedom are n − 1.
• Confusing standard deviation (spread of data) with standard error (spread of the mean) — the commonest exam slip.
• Reading causation into a correlation between two plant tags that are both driven by throughput.
• Reconciling data without first testing for gross errors, so one faulty meter corrupts every reconciled flow.
• Averaging highly autocorrelated DCS samples and claiming a tiny confidence interval from a large but non-independent n.

Related topics

Material balances · Energy balances · Process instrumentation and measurement · Process control and dynamics · Reaction kinetics and reactor design · Process optimisation and simulation