Medical image processing

In short

Medical image processing is the set of computational steps that turn raw detector data from X-ray, CT, MRI, ultrasound and nuclear scanners into images a radiologist or an algorithm can read reliably - reconstruction, enhancement, restoration, segmentation, registration, quantification, compression and storage. It matters because the raw signal is noisy, low in contrast and geometrically distorted; diagnosis depends on what processing recovers and on not creating artefacts that look like disease. In Indian universities it is a core paper in Biomedical/Medical Electronics (and appears in GATE BM as "image enhancement, segmentation, registration, reconstruction"), and employers - Philips Healthcare Bengaluru, GE Healthcare, Siemens Healthineers, Wipro GE, Qure.ai, Niramai, DeepTek, HCL/TCS medical-device divisions, AIIMS/SCTIMST research groups - interview on exactly these steps. The regulatory frame is the Medical Devices Rules, 2017 under CDSCO, which bring software intended for diagnosis in as a medical device, and the DICOM standard for every image file and network transfer.

Explained

Digital image fundamentals and the medical image

• A digital image is a 2-D function f(x,y) sampled on a grid of pixels; a volume is sampled as voxels with a slice thickness, so CT/MRI data are inherently 3-D.
• Spatial resolution = pixel size and point-spread function; contrast resolution = number of distinguishable grey levels; temporal resolution matters in fluoroscopy, cine-MRI and echo.
• Bit depth: CT stores 12 bits (Hounsfield units −1024 to +3071), digital radiography 12-16 bits, ultrasound and fundus images 8 bits per channel. Displays show only ~256 grey levels, hence windowing.
• Windowing (level/width): a linear grey-level transform that maps a chosen HU interval to the full display range - window level WL is the centre, window width WW the span. Lung, bone, soft-tissue and brain windows are presets.
• Sampling and aliasing: Nyquist requires sampling at ≥ twice the highest spatial frequency; undersampling in MRI phase-encode direction causes wrap-around (fold-over) aliasing.
• DICOM (Digital Imaging and Communications in Medicine, NEMA PS3): file format + network protocol. An object = pixel data + a header of tags (patient, study, series, instance UIDs; pixel spacing; rescale slope/intercept; modality). Services: C-STORE, C-FIND, C-MOVE, Modality Worklist; SOP classes identify object types.
• HU conversion: stored values are rescaled - HU = slope × stored + intercept (typically slope 1, intercept −1024). Forgetting this is a classic bug.
• PACS/VNA: Picture Archiving and Communication System stores and distributes DICOM; HL7/FHIR carries the text (orders, reports) alongside; RIS schedules. India's ABDM ecosystem is pushing standards-based health records; many government hospitals run PACS under NMC/State tenders.

Image enhancement (spatial and frequency domain)

• Point operations: negative s = L−1−r (useful for mammograms); log transform to compress dynamic range; power-law (gamma) s = c·r^γ; contrast stretching; grey-level slicing; bit-plane slicing.
• Histogram equalisation: redistributes grey levels using the cumulative distribution function so the output histogram is near-uniform - boosts global contrast but amplifies noise and can produce unnatural appearance.
• AHE / CLAHE: adaptive (tile-wise) histogram equalisation; CLAHE clips the histogram before equalising to limit noise amplification - the standard choice for chest X-ray, retinal and dermoscopic images.
• Spatial smoothing filters: mean/box (blurs edges), Gaussian (weighted, smooth frequency roll-off), median (order-statistic, removes salt-and-pepper and ultrasound speckle spikes while preserving edges - non-linear, so not a convolution).
• Edge-preserving filters: bilateral (range + domain weights), anisotropic diffusion (Perona-Malik), non-local means - preferred in low-dose CT and MRI denoising.
• Sharpening: Laplacian g = f − ∇²f; unsharp masking / high-boost g = f + k(f − blurred f).
• Frequency domain: 2-D DFT, multiply by a transfer function H(u,v), inverse DFT. Ideal low-pass rings (Gibbs), Butterworth and Gaussian low-pass smooth; high-pass and homomorphic filtering correct illumination/bias field.
• MRI bias-field (intensity inhomogeneity) correction: N4ITK is the standard, applied before any intensity-based segmentation.

Noise, restoration and reconstruction

• Noise models: Poisson (quantum/photon noise in X-ray, CT, PET/SPECT - variance = mean), Gaussian (electronic), Rician (MRI magnitude images at low SNR), multiplicative speckle (ultrasound, coherent interference), salt-and-pepper (dead/hot detector elements → ring or line artefacts in CT).
• Restoration assumes a degradation model g = h ∗ f + n and tries to invert it: inverse filter (unstable where H→0), Wiener filter (minimum mean-square error, needs noise-to-signal power ratio), constrained least squares, Richardson-Lucy (Poisson, used in SPECT/PET).
• CT reconstruction - filtered back projection (FBP): projections p(s,θ) are the Radon transform of the object; the Fourier slice theorem says the 1-D FT of a projection equals a radial line through the 2-D FT of the object. Steps: take 1-D FT of each projection → multiply by |ω| ramp filter (optionally Shepp-Logan/Hann window to suppress noise) → inverse 1-D FT → back-project over all θ. Without the ramp, the image is 1/r blurred.
• Iterative reconstruction: ART/SART, MLEM and OSEM (ordered-subset expectation maximisation, standard in PET/SPECT), and model-based IR in CT (Siemens SAFIRE, GE ASiR/Veo, Philips iDose) - cut dose 30-70% at the same noise, at higher compute cost and a changed "plastic" noise texture.
• MRI reconstruction: raw data are samples of k-space; image = 2-D inverse FT. Centre of k-space carries contrast, periphery carries detail. Partial Fourier, parallel imaging (SENSE in image domain, GRAPPA in k-space) and compressed sensing (incoherent undersampling + sparsity + non-linear recovery) shorten scans.
• Common artefacts to recognise: CT beam hardening and metal streaks, ring artefact (miscalibrated detector), motion ghosting (MRI phase direction), chemical shift, susceptibility, Gibbs/truncation ringing, ultrasound reverberation and shadowing, PET attenuation-correction mismatch.

Segmentation, morphology and feature extraction

• Thresholding: global (Otsu - maximises between-class variance), adaptive/local (Niblack, Sauvola) for uneven illumination, multi-level thresholding for CT tissue classes.
• Region-based: region growing from seeds (needs homogeneity criterion), split-and-merge (quadtree), watershed on the gradient image (over-segments unless markers are used).
• Edge-based: gradient operators Sobel/Prewitt/Roberts, Laplacian of Gaussian (zero-crossings), Canny (smooth → gradient → non-maximum suppression → hysteresis double threshold).
• Model-based: active contours/snakes (energy = internal smoothness + external image force), level sets (handle topology change), atlas-based segmentation for brain structures, statistical shape models.
• Clustering/statistical: k-means, fuzzy c-means (handles partial-volume tissue mixing), Gaussian mixture models with expectation maximisation, Markov random fields for spatial smoothness.
• Morphological operations on binary/grey images: erosion, dilation, opening (erode then dilate - removes small objects, breaks bridges), closing (fills holes/gaps), hit-or-miss, top-hat (uneven background removal), skeletonisation, connected-component labelling to count and label lesions.
• Features: shape (area, perimeter, circularity, eccentricity), intensity statistics, texture (GLCM contrast/homogeneity/energy/entropy, run-length, LBP, Gabor, wavelet), Hu moments (invariant to translation, rotation, scale). "Radiomics" is high-throughput extraction of such features for prognosis.
• Validation metrics: Dice similarity coefficient, Jaccard/IoU, Hausdorff distance, sensitivity/specificity, and ROC/AUC for detection - against an expert manual ground truth with inter-observer variability stated.

Registration, fusion, compression and visualisation

• Registration aligns two images: components are a transform (rigid 6-DOF, affine 12-DOF, or non-rigid B-spline/demons), a similarity measure (SSD for same modality, normalised cross-correlation, mutual information for multi-modal), an interpolator (nearest, linear, B-spline) and an optimiser (gradient descent, Powell), usually run coarse-to-fine on a pyramid.
• Uses: PET-CT and PET-MR fusion, pre/post-treatment comparison, atlas mapping, motion correction in fMRI/DSA, image-guided surgery and neuronavigation, radiotherapy planning (CT for dose, MRI for target).
• Digital subtraction angiography (DSA): mask image subtracted from contrast image after registration - pure vessel map; pixel shifting corrects patient motion.
• Compression: lossless (RLE, Huffman, LZW, JPEG-LS, JPEG 2000 lossless) is the normal practice for primary diagnostic archives; lossy (JPEG, JPEG 2000) only where locally permitted and labelled - the DICOM header records the transfer syntax. Measure with compression ratio, MSE, PSNR, SSIM.
• Visualisation: multiplanar reformatting (MPR), maximum intensity projection (MIP for angiography), surface rendering, volume rendering with transfer functions, curved reformat for vessels, 3-D printing/STL export for surgical planning.
• CAD and AI: classical CAD (feature + classifier: SVM, random forest) has largely been replaced by CNNs - U-Net and its variants for segmentation, ResNet/DenseNet/EfficientNet for classification, YOLO/Faster R-CNN for detection, transformers and foundation models more recently. Practical issues: limited labelled data (augmentation, transfer learning), class imbalance (Dice/focal loss), domain shift between scanners, and explainability (Grad-CAM).

Indian standards, regulation and quality

• CDSCO / Medical Devices Rules, 2017 (under the Drugs and Cosmetics Act, 1940 framework; all medical devices were notified as drugs with effect from 1 April 2020, with licensing mandatory for Class A/B from 1 October 2022 and Class C/D from 1 October 2023): software intended for diagnosis or treatment decisions is a medical device. Risk classes A (low) to D (high); diagnostic image-analysis software is typically Class B or C and needs a manufacturing/import licence - manufacturing licences come from the State Licensing Authority for Class A/B and the Central Licensing Authority for Class C/D, while all import licences are central. Clinical/performance evidence follows the New Drugs and Clinical Trials Rules, 2019 and the device rules.
• Quality systems: ISO 13485 for medical-device QMS; IEC 62304 for medical device software life-cycle; IEC 62366 usability; ISO 14971 risk management; IEC 60601-1 and 60601-2-xx for the imaging equipment itself. ISO 9001:2026 is the current general QMS edition but ISO 13485 is the device-specific one. Most are voluntary standards that become contractually or regulatorily expected rather than statutory on their own.
• Standards bodies: BIS publishes Indian adoptions of several IEC/ISO device standards (quote by current number only if verified - many older IS numbers have been superseded); AERB regulates the radiation-producing equipment and licensing of X-ray/CT/nuclear-medicine installations, including acceptance and QA tests that are image-quality tests in practice.
• Image-quality QA: resolution phantoms (line pairs/mm), low-contrast detectability, CT number accuracy and uniformity with a water phantom (water = 0 ± 5 HU typically), noise SD, slice thickness, MTF and NPS measurement; for displays, the DICOM Part 14 GSDF calibration of diagnostic monitors.
• Data protection: the Digital Personal Data Protection Act, 2023 with the DPDP Rules, 2025 - the Data Fiduciary duties (notice, consent, breach intimation, erasure) and the omission of IT Act section 43A apply only from 13 May 2027; until then IT Act section 43A and the SPDI Rules, 2011 govern sensitive personal data including health data. De-identification of DICOM for research means stripping patient tags and also burnt-in pixel annotations (common in ultrasound) - DICOM Supplement 142 profiles cover this.
• Teleradiology Practice Guidelines (2020, MoHFW/NMC Telemedicine framework) govern remote reporting; image compression and display quality are part of that responsibility.

Key points

• Pixel/voxel - smallest 2-D/3-D sample; image quality is set by pixel size, slice thickness and the system point-spread function.
• Windowing (WL/WW) - linear mapping of a HU range onto the display; changes appearance only, never the stored data.
• Hounsfield unit - CT density scale, water = 0, air = −1000, dense bone ≈ +1000 or more; always apply rescale slope/intercept.
• Histogram equalisation - CDF-based global contrast boost; CLAHE is the tile-wise, noise-limited version used in practice.
• Median filter - non-linear, best for salt-and-pepper and speckle spikes, preserves edges; mean/Gaussian blur edges.
• Wiener filter - MMSE restoration needing the degradation PSF and noise-to-signal ratio; inverse filtering alone explodes noise.
• Fourier slice theorem - 1-D FT of a projection = a radial slice of the object's 2-D FT; the basis of FBP.
• Ramp (|ω|) filter - corrects the 1/r blur of simple back-projection; windowed (Hann, Shepp-Logan) to control noise.
• OSEM/iterative reconstruction - statistical, models noise and physics, lowers dose, costs compute, changes noise texture.
• k-space - MRI raw data domain; centre = contrast, edges = detail; image = inverse 2-D FT.
• Otsu thresholding - automatic global threshold maximising between-class variance; fails on bimodal-less or shaded images.
• Canny edge detector - smooth, gradient, non-maximum suppression, hysteresis thresholding; best single edge detector in exams.
• Opening vs closing - opening removes small bright specks and thin bridges; closing fills small holes and gaps.
• Mutual information - intensity-relationship similarity measure that makes multimodal (PET-CT, MR-CT) registration possible.
• Dice coefficient - 2|A∩B|/(|A|+|B|), the standard segmentation accuracy score; Hausdorff distance reports worst boundary error.
• Lossless compression for diagnostic archives; lossy only if labelled and locally permitted - the DICOM transfer syntax records it.
• U-Net - encoder-decoder CNN with skip connections, the default medical-image segmentation architecture.
• CDSCO MDR 2017 + IEC 62304 - diagnostic software is a regulated medical device needing a documented software life-cycle and risk file.

Formulas and examples

Example 1 - HU and windowing. A CT voxel has μ = 0.206 cm⁻¹; water μ = 0.190 cm⁻¹.

• HU = 1000 × (0.206 − 0.190)/0.190 = 1000 × 0.016/0.190 = +84 HU (soft tissue/haematoma range).
• Displayed in a brain window WL = 40, WW = 80: range is 0 to 80 HU. Output = 255 × (84 − 0)/80 > 255 → pure white (saturated).
• In a soft-tissue window WL = 50, WW = 350: lower edge = −125, output = 255 × (84 + 125)/350 = 255 × 0.5971 ≈ 152 (mid-grey) - the same data, two appearances.

Example 2 - study size and transfer time. A chest CT: 512 × 512 pixels, 400 slices, 16 bits stored, single channel.

• Raw size = 512 × 512 × 400 × 2 bytes = 209,715,200 bytes ≈ 200 MiB (209.7 MB).
• JPEG 2000 lossless at CR = 2.5 → ≈ 83.9 MB; at lossy CR = 10 → ≈ 21.0 MB.
• Over a 20 Mbit/s teleradiology link, the lossless file takes 83.9 × 8 / 20 ≈ 33.6 s; the raw study ≈ 83.9 s. This is why compression and streaming matter in Indian district-hospital links.

Example 3 - histogram equalisation (3-bit image, L = 8, N = 64 pixels).

• Check: 8+10+16+14+10+4+2 = 64. Levels 5, 6 and 7 merge into 7 - equalisation can *reduce* the number of distinct levels while spreading them over the full range.

Example 4 - segmentation scoring. An automated liver segmentation gives 86,000 voxels; the radiologist's ground truth has 90,000; the overlap is 82,000.

• Dice = 2 × 82,000 / (86,000 + 90,000) = 164,000/176,000 = 0.932.
• Jaccard = 82,000 / (86,000 + 90,000 − 82,000) = 82,000/94,000 = 0.872. (Cross-check: 2J/(1+J) = 1.7447/1.8723 = 0.932 ✓)
• Sensitivity = 82,000/90,000 = 91.1%; false-positive voxels = 86,000 − 82,000 = 4,000.

Example 5 - denoising, SNR and PSNR. A region of interest in an MR image has mean 540 and background noise SD 18; after a 3×3 mean filter the noise SD falls by √9 = 3 (ideal, uncorrelated noise) to 6.

• SNR before = 540/18 = 30; after = 540/6 = 90 (+9.5 dB), but the point-spread width roughly triples, so fine detail is lost.
• If the filtered image differs from the clean reference with MSE = 25 on an 8-bit scale, PSNR = 10 log₁₀(255²/25) = 10 log₁₀(2601) = 34.2 dB - typically "visually good" for 8-bit images.

Where it is used

• Imaging OEM R&D in India: Philips Innovation Campus and GE Healthcare (Bengaluru), Siemens Healthineers, Samsung Medison, Mindray and Allengers (Chandigarh) - reconstruction, artefact correction, workflow and QA software roles.
• AI health-tech startups: Qure.ai (chest X-ray and head CT triage, deployed in TB screening programmes under the National TB Elimination Programme), DeepTek, Niramai (thermal breast screening), Predible, Artelus, SigTuple - CV/ML engineer and clinical-validation roles.
• Hospital/PACS and telemedicine: PACS administration and DICOM integration in Apollo, Fortis, AIIMS and state health systems; teleradiology reporting hubs; ABDM-linked record exchange.
• Radiotherapy and surgical planning: contouring and MR-CT registration for treatment planning (Eclipse, Monaco), neuronavigation, orthopaedic 3-D printing from CT at centres such as SCTIMST Thiruvananthapuram.
• Regulatory, QA and service: CDSCO/notified-body documentation under MDR 2017 and IEC 62304, AERB-linked acceptance and periodic QA of CT/X-ray units, display GSDF calibration, medical-physicist roles in hospitals.
• Academia and research: C-DAC, IITs, IISc, IIT Madras healthcare technology centres, SCTIMST, NIMHANS brain-imaging labs; ophthalmic (retinal) and pathology whole-slide image analysis.

Interview questions

• Q: Why is a ramp filter needed in filtered back projection? A: Simple back-projection over-weights low spatial frequencies and gives a 1/r blur; the |ω| ramp compensates that weighting in the frequency domain, and it is usually windowed (Hann/Shepp-Logan) to stop the high-frequency noise it amplifies.
• Q: When would you use a median filter instead of a Gaussian filter? A: When the noise is impulsive - salt-and-pepper or speckle spikes - because the median is an order statistic that rejects outliers while keeping edges sharp, whereas a Gaussian averages the outlier into the neighbourhood.
• Q: What does windowing actually change? A: Only the display mapping of stored values to grey levels; the pixel data are untouched, which is why a lesion invisible in a bone window appears in a soft-tissue window.
• Q: Why is mutual information used for PET-CT registration? A: The two modalities have no fixed intensity relationship, so SSD or correlation fail; mutual information only requires that the joint intensity histogram becomes statistically more predictable at correct alignment.
• Q: What is k-space and why is its centre important? A: k-space holds the spatial-frequency samples acquired in MRI; the centre encodes low frequencies - overall contrast and signal - while the periphery encodes edges and fine detail, which is why partial-Fourier and centric ordering work.
• Q: Histogram equalisation gave a worse-looking chest X-ray - why, and what would you use? A: Global equalisation amplifies noise and over-stretches a dominant background; CLAHE applied tile-wise with a clip limit gives local contrast without the noise blow-up.
• Q: Difference between image enhancement and image restoration? A: Enhancement is subjective and heuristic (make it look better for a viewer); restoration is objective and model-based, inverting a known degradation such as blur plus noise.
• Q: Why is lossy compression controversial in radiology? A: It can remove low-contrast detail such as a subtle lesion or microcalcification; diagnostic archives therefore use lossless or mathematically-verified compression, and any lossy step must be recorded in the DICOM transfer syntax and allowed by local policy.
• Q: Your segmentation model scores Dice 0.93 in the lab but 0.78 at a client hospital - why? A: Domain shift - different scanner, kernel, slice thickness, contrast protocol and population; fix with intensity normalisation, scanner-diverse training data, augmentation and site-specific fine-tuning.
• Q: What is the difference between opening and closing? A: Opening is erosion followed by dilation and removes small bright structures and thin connections; closing is dilation followed by erosion and fills small dark holes and gaps - both approximately preserve object size.
• Q: How do you de-identify a DICOM study for a research dataset? A: Remove or replace patient-identifying tags per the DICOM confidentiality profiles, re-map UIDs consistently, strip private tags, and check for burnt-in pixel annotations - common in ultrasound and screen captures - which tag removal will not touch.
• Q: Is an AI image-analysis tool a medical device in India? A: Yes - under the Medical Devices Rules, 2017 software intended for diagnosis or treatment decisions is regulated by CDSCO with a risk class (usually B or C for diagnostic imaging software) and requires a licence plus documented performance evidence and an IEC 62304-style software life-cycle.

Common mistakes

• Reading DICOM pixel values as HU without applying rescale slope and intercept, or ignoring signed/unsigned pixel representation - every density calculation then comes out shifted by about 1024.
• Treating the median filter as a convolution, or trying to "separate" it like a Gaussian kernel - it is non-linear and neither associative with convolution nor separable.
• Writing histogram equalisation as a stretch of min-max instead of a CDF mapping, and claiming the output histogram is exactly flat (it is only approximately uniform for discrete levels, and levels merge).
• Omitting the ramp filter or applying it after back-projection in FBP derivations; also confusing the Radon transform with the Fourier transform in the slice-theorem statement.
• Reporting segmentation accuracy as pixel accuracy on a small lesion - a model that outputs all background can score 99% accuracy with Dice 0; always report Dice/IoU and the class balance.
• Smoothing before measuring resolution or SNR, then quoting the improved SNR without saying that spatial resolution was traded away.
• Calling anything "FDA/CDSCO approved" loosely - state the actual class, licence and intended use, and never claim a certification is mandatory unless verified.

Related topics

X-ray imaging basics · CT and MRI basics · Ultrasound basics · Nuclear medicine imaging: PET and SPECT · Digital signal processing and transforms · Biomedical instrumentation and medical device regulation