Convolutional neural networks

In short

A convolutional neural network is a neural network in which most layers replace full connectivity with a small learnable kernel slid across the input, so that nearby pixels are combined locally and the same weights are reused at every position. This gives three properties a plain dense network lacks — local connectivity, weight sharing and built-in equivariance to translation — and cuts parameters by orders of magnitude on image-sized inputs. CNNs are the standard model for images, video, spectrograms and 1-D signals, and remain the deployed choice on edge hardware even where transformers win on accuracy. In India they are examined in GATE DA/CS (deep learning section), in university AI&DS courses as the core vision unit, and asked in interviews at Qualcomm, Bosch, Samsung R&D, Tata Elxsi, Nvidia, ISRO/NRSC, Qure.ai, Niramai and most computer-vision startups. Numerical questions on output size, parameter count and receptive field are near-certain in both exams and screening tests.

Explained

The convolution layer

• Operation: a kernel (filter) of size k×k×C_in is slid over the input volume; at each position the element-wise product is summed and a bias added, giving one scalar in one output feature map. C_out filters give C_out maps.
• Cross-correlation, not true convolution: frameworks do not flip the kernel. True convolution flips it (f∗g); since the kernel is learnt, the distinction only matters in derivations and in the backward pass.
• Hyperparameters: kernel size k, number of filters C_out, stride S (step of the slide; S>1 downsamples), padding P (zeros added around the border), dilation d (gaps inside the kernel).
• Padding types: *valid* (P=0, output shrinks), *same* (P chosen so output size = input size when S=1, i.e. P=(k−1)/2 for odd k), *full* (P=k−1, output grows). Odd kernels are preferred so that "same" padding is symmetric.
• Weight sharing: the same k×k×C_in weights are used at all spatial positions — hence far fewer parameters, and a feature detected in one corner is detected anywhere.
• Equivariance vs invariance: convolution is *equivariant* to translation (shift the input, the feature map shifts). Invariance comes from pooling, striding and global pooling — and is only approximate. CNNs are not inherently invariant to rotation or scale; that is supplied by augmentation or special architectures.
• 1×1 convolution: mixes channels at a fixed pixel; used to cut or expand channel depth cheaply (bottlenecks in Inception/ResNet) and as a learnable per-pixel linear layer.
• Dilated (atrous) convolution: inserts d−1 gaps, giving effective kernel k+(k−1)(d−1) and a large receptive field without extra parameters or loss of resolution — used in segmentation (DeepLab) and audio (WaveNet).
• Grouped convolution: input channels split into g groups, each with its own filters (AlexNet's two-GPU split; ResNeXt). Depthwise convolution is the extreme g = C_in.
• Transposed / fractionally-strided convolution ("deconvolution"): upsamples; used in decoders, segmentation and generators. Checkerboard artefacts appear when k is not a multiple of S.

Pooling, activation and the rest of the stack

• Max pooling: takes the maximum in each window (typically 2×2, S=2) — keeps the strongest activation, adds small-shift tolerance, no parameters. Average pooling smooths instead.
• Backward pass of pooling: max pooling routes the gradient only to the arg-max position (needs the stored index); average pooling spreads it equally.
• Global average pooling (GAP): averages each map to one number, producing a C-vector. Replaces the huge flatten+dense head (as in NiN, ResNet), removes most parameters, allows variable input size and underpins class activation maps (CAM).
• Activation: ReLU is the default inside CNNs (cheap, non-saturating); variants LeakyReLU, ELU, GELU, SiLU/Swish, and hard-swish on mobile. Applied element-wise after convolution (and after normalisation).
• Normalisation in CNNs: batch norm is applied per channel across (N,H,W) — 2C learnable parameters (γ, β) per layer. Group/instance/layer norm are used when batch size is tiny or style transfer demands per-sample statistics.
• Typical block: Conv → Norm → Activation, repeated, with downsampling by pooling or stride 2, then GAP/flatten → dense → softmax.
• Standard full pipeline: input normalisation → feature extractor (stacked conv blocks, channels doubling as resolution halves) → classifier head → cross-entropy loss.
• Data augmentation (vision-specific): random crop, horizontal flip, scale/rotation, colour jitter, Cutout, Mixup, CutMix, RandAugment. Flips must not be used where orientation carries meaning (text, medical laterality, road signs).

Receptive field and the arithmetic of depth

• Receptive field (RF): the region of the *input* that influences one output unit. Recurrence: RF_l = RF_{l−1} + (k_l − 1) × J_{l−1}, where the jump J_l = J_{l−1} × S_l, starting RF₀ = 1, J₀ = 1.
• Stacking small kernels is better than one large kernel: three 3×3 layers give RF 7 with 27C² weights against 49C² for one 7×7, plus two extra non-linearities (the VGG argument).
• RF grows linearly with depth for stride-1 convolutions, and multiplicatively once strides/pooling are added — which is why detection and segmentation networks need strides, dilation or pyramids to "see" large objects.
• Effective RF is smaller than the theoretical RF and roughly Gaussian in weight; central pixels dominate.

How a CNN is trained (what is specific to convolution)

1. Forward: Z = W ∗ X + b per layer; cache X and pooling indices.
2. Loss: cross-entropy for classification; IoU/Dice-type losses for segmentation; smooth-L1 + classification for detection.
3. Gradient to weights: ∂L/∂W is itself a convolution of the input patch with the output gradient — each weight accumulates contributions from every spatial position (consequence of weight sharing).
4. Gradient to input: ∂L/∂X is a *full* convolution of ∂L/∂Z with the 180°-rotated kernel — equivalently a transposed convolution. Stride S in the forward pass becomes dilation of the gradient map in the backward pass.
5. Bias gradient: sum of ∂L/∂Z over all positions of that channel.
6. Implementation: im2col + GEMM (BLAS/cuBLAS), FFT for large kernels, Winograd for 3×3 — the reason 3×3 is unusually fast on GPUs and NPUs.
7. Initialisation: He/Kaiming for ReLU nets, with fan_in = k²·C_in. Xavier/Glorot for tanh.

Landmark architectures (what each contributed)

• Why skip connections work: they give an identity path, so gradients reach early layers undiminished and a block only has to learn a residual — the standard answer to "why can we train 100+ layers".
• Task heads: classification (GAP + softmax); detection — two-stage (R-CNN, Fast/Faster R-CNN with RPN and RoI pooling) vs one-stage (YOLO, SSD, RetinaNet with focal loss); segmentation — semantic (FCN, U-Net, DeepLab) vs instance (Mask R-CNN); keypoints, depth, super-resolution.
• Evaluation metrics: top-1/top-5 accuracy; IoU and mAP@0.5 / mAP@[.5:.95] for detection; mIoU and Dice for segmentation.

Advantages, limitations and comparisons

• Advantages: few parameters for image-sized inputs; translation equivariance; strong inductive bias so they learn from modest datasets; hierarchical features (edges → textures → parts → objects); highly optimised on hardware; fixed cost independent of sequence length.
• Limitations: limited receptive field per layer (global context needs depth or dilation); no built-in rotation/scale invariance; fixed-size dense heads tie the model to one input size; poor at long-range relational reasoning; vulnerable to adversarial perturbations and to texture bias; pooling discards precise localisation.
• Versus fully connected: a dense layer on a 224×224×3 image with 1,000 hidden units needs ~150 million weights and ignores spatial structure; a conv layer needs thousands and reuses them.
• Versus transformers (covered separately): CNNs have a stronger local prior and are more data- and compute-efficient at small scale; attention-based models scale better with data and model global context natively. Hybrids (ConvNeXt, CoAtNet, MobileViT) are common in production.
• 1-D and 3-D convolution: Conv1D for ECG, speech features, sensor and NLP character models; Conv3D and (2+1)D for video and CT/MRI volumes.
• Deployment: quantisation to INT8, pruning, knowledge distillation; runtimes ONNX Runtime, TensorRT, TFLite, OpenVINO. On Indian edge projects (ANPR cameras, agri drones) INT8 CNNs on Jetson/Snapdragon/Hailo are the norm.
• Regulatory note (India): vision models on personal data — faces, biometrics, patient scans — fall under the Digital Personal Data Protection Act, 2023 (rules notified in 2025; phased compliance), which makes consent, purpose limitation and data-fiduciary duties mandatory; CNN-based diagnostic software is regulated as a medical device by CDSCO under the Medical Devices Rules, 2017, and ICMR's 2022 ethical guidelines for AI in healthcare apply. Confirm current notification status before relying on specific dates.

Key points

• Convolution layer — small learnable kernel slid over the input; local connectivity plus weight sharing.
• Output size — floor((W − k + 2P)/S) + 1 per spatial dimension, independent of C_in.
• Parameters — (k·k·C_in + 1)·C_out; completely independent of input H and W.
• Same padding — P = (k−1)/2 with S = 1 preserves spatial size; valid padding shrinks it by k−1.
• Depth of a kernel — always equals the input channel count; a "3×3 filter" on RGB is really 3×3×3.
• Pooling — parameter-free downsampling; max keeps the strongest response, average smooths.
• Global average pooling — replaces the flatten+dense head, removes millions of weights, enables any input size.
• 1×1 convolution — channel mixing / dimensionality change at one pixel; the bottleneck trick.
• Receptive field — RF_l = RF_{l−1} + (k_l − 1)·∏S; grows fast once strides and pooling are added.
• Small stacked kernels — three 3×3 ≡ RF of one 7×7 with ~45% fewer weights and more non-linearity.
• Dilated convolution — effective kernel k+(k−1)(d−1); bigger context, same parameters, same resolution.
• Depthwise separable convolution — depthwise + pointwise; cost ratio 1/C_out + 1/k² of a standard conv.
• Residual connection — y = F(x) + x; identity gradient path that makes very deep CNNs trainable.
• Transposed convolution — learnable upsampling; output = (W−1)S − 2P + k (+ output padding).
• Batch norm in CNNs — statistics per channel over (N,H,W); 2C learnable parameters.
• Equivariance vs invariance — convolution is translation-equivariant; invariance to rotation/scale must be trained in.
• Cost driver — early layers dominate FLOPs (large H×W), late layers and dense heads dominate parameters.
• Backward pass of convolution — convolution with the flipped kernel; weight gradients summed over all positions.

Formulas and examples

Example 1 — Output size, parameters and MACs of one layer. Input 32×32×3 (CIFAR-10); 64 filters of 3×3, S = 1, P = 1, bias on.

• H_out = (32 − 3 + 2)/1 + 1 = 32 → output 32×32×64.
• Parameters = (3·3·3 + 1)·64 = 28 × 64 = 1,792.
• MACs = 32·32·64 × 27 = 65,536 × 27 = 1,769,472 ≈ 1.77 M (≈3.54 MFLOPs).
• Activation memory = 65,536 × 4 = 262,144 bytes ≈ 256 KB per image.

Example 2 — AlexNet first layer and the pooling that follows. Input 227×227×3; 96 filters 11×11, S = 4, P = 0.

• H_out = (227 − 11)/4 + 1 = 216/4 + 1 = 55 → 55×55×96.
• Parameters = (11·11·3 + 1)·96 = 364 × 96 = 34,944.
• MACs = 55·55·96 × 363 = 290,400 × 363 = 105,415,200 ≈ 105.4 M.
• Max pool 3×3, S = 2: (55 − 3)/2 + 1 = 27 → 27×27×96, 0 parameters.

Note the asymmetry: this layer has 35 k weights but 105 M MACs, while AlexNet's first dense layer has ~37 M weights and only ~37 M MACs.

Example 3 — Full LeNet-style network on MNIST; total parameters. Input 28×28×1.

• Total = 156 + 2,416 + 30,840 + 10,164 + 850 = 44,426 parameters, of which 41,854 (94%) sit in the dense head — the motivation for global average pooling.
• Replacing Flatten+FC1+FC2 with GAP (16 features) + FC(10) gives 16·10+10 = 170, total 2,742 parameters.

Example 4 — Receptive field of a small stack. Layers: Conv 3×3 S1 → MaxPool 2×2 S2 → Conv 3×3 S1 → MaxPool 2×2 S2 → Conv 3×3 S1.

• Start RF = 1, J = 1.
• Conv: RF = 1 + 2·1 = 3, J = 1. Pool: RF = 3 + 1·1 = 4, J = 2.
• Conv: RF = 4 + 2·2 = 8, J = 2. Pool: RF = 8 + 1·2 = 10, J = 4.
• Conv: RF = 10 + 2·4 = 18 pixels, J = 4 (one output step = 4 input pixels).

With dilation d = 2 in the last conv (k_eff = 5): RF = 10 + 4·4 = 26 pixels, parameters unchanged.

Example 5 — Standard vs depthwise separable convolution. Input 14×14×256, output 14×14×256, k = 3, S = 1, P = 1, no bias.

• Standard: parameters = 3·3·256·256 = 589,824; MACs = 14·14·256 × (9·256) = 50,176 × 2,304 = 115,605,504 ≈ 115.6 M.
• Depthwise (3×3, one filter per channel) = 9 × 256 = 2,304; pointwise (1×1, 256→256) = 65,536 → total 67,840.
• MACs: depthwise = 196·256·9 = 451,584; pointwise = 196·256·256 = 12,845,056 → 13.30 M.
• Ratio = 67,840/589,824 = 0.115, i.e. ≈8.7× fewer parameters and MACs — matching 1/C_out + 1/k² = 1/256 + 1/9 = 0.115.

Example 6 — Upsampling in a decoder. Transposed conv, input 7×7, k = 4, S = 2, P = 1: H_out = (7−1)·2 − 2 + 4 = 14. With k = 3, S = 2, P = 1, output_padding = 1: (7−1)·2 − 2 + 3 + 1 = 14 (exact 2× upsampling).

import torch, torch.nn as nn
block = nn.Sequential(
    nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False),  # 'same'
    nn.BatchNorm2d(64), nn.ReLU(inplace=True),
    nn.MaxPool2d(2, 2),                                                # 32 -> 16
    nn.Conv2d(64, 128, 3, padding=1, bias=False),
    nn.BatchNorm2d(128), nn.ReLU(inplace=True),
    nn.AdaptiveAvgPool2d(1), nn.Flatten(), nn.Linear(128, 10))
x = torch.randn(8, 3, 32, 32)
print(block(x).shape)                                     # torch.Size([8, 10])
print(sum(p.numel() for p in block.parameters()))         # 77,130

Where it is used

• Space and agriculture: ISRO/NRSC and agritech firms run CNNs (U-Net, segmentation backbones) on Resourcesat/Cartosat and drone imagery for crop classification, acreage estimation under schemes like PM-Fasal Bima claims, flood and encroachment mapping.
• Healthcare: Qure.ai (qXR chest X-ray TB screening used in state TB-elimination programmes), Niramai thermal breast screening, Aindra, Tricog; CNN-based diagnostic software is a CDSCO-regulated medical device, and Indian hospital deployments require clinician-in-the-loop validation.
• Mobility and surveillance: ANPR at NHAI/FASTag toll plazas and city traffic enforcement, Smart Cities Mission CCTV analytics, railway track and coach inspection, ADAS stacks at Tata Elxsi, KPIT, Bosch, Continental and Mercedes-Benz R&D Bengaluru.
• Manufacturing and inspection: visual defect detection in textiles, pharma blister packs, steel surfaces and PCB assembly; roles in Industry-4.0 teams at L&T, Tata Steel, Siemens, Flex.
• Banking, fintech and identity: cheque/KYC document OCR, Aadhaar-based face authentication, signature verification, insurance motor-claim damage assessment — all under DPDP Act, 2023 obligations for biometric data.
• Job roles: computer vision engineer, deep learning engineer, edge-AI/embedded-AI engineer (TensorRT, TFLite, OpenVINO), ML research engineer, imaging data scientist; GATE DA, ISRO/DRDO/C-DAC and PSU recruitment tests examine the layer arithmetic directly.

Interview questions

• Q: Why use convolution instead of a fully connected layer on images? A: Local connectivity plus weight sharing cut parameters by orders of magnitude, exploit the spatial locality of pixels and make feature detection translation-equivariant, so the model generalises from far less data.
• Q: Write the output-size formula and apply it to 64×64 input, 5×5 kernel, stride 2, padding 2. A: ⌊(W − k + 2P)/S⌋ + 1 = ⌊(64 − 5 + 4)/2⌋ + 1 = ⌊31.5⌋ + 1 = 32, so output is 32×32 per filter.
• Q: Do the parameters of a conv layer depend on the input image size? A: No — only on k, C_in and C_out. Only activation memory and FLOPs scale with H and W, which is why a conv backbone accepts variable input sizes while a flatten+dense head does not.
• Q: Why stack 3×3 convolutions instead of using 7×7? A: Three 3×3 layers have the same 7×7 receptive field but 27C² weights against 49C², and add two extra non-linearities, so they are cheaper and more expressive.
• Q: What does a 1×1 convolution achieve if it covers only one pixel? A: It is a learnable linear projection across channels — used to reduce or expand depth cheaply (bottlenecks), add non-linearity and mix feature maps without touching spatial resolution.
• Q: How does the gradient flow through max pooling? A: Only to the position that produced the maximum (stored as an index); all other positions in the window get zero gradient.
• Q: Difference between equivariance and invariance in a CNN? A: Convolution is equivariant — shifting the input shifts the feature map; invariance (the output not changing) comes approximately from pooling, striding and global pooling, and is not automatic for rotation or scale.
• Q: Why do residual connections let us train very deep CNNs? A: The identity path passes gradients to early layers without attenuation and lets each block learn only a residual correction, so adding depth cannot make optimisation harder.
• Q: What is a dilated convolution and when would you use it? A: A convolution with gaps in the kernel, effective size k+(k−1)(d−1); it enlarges the receptive field for dense tasks like segmentation without downsampling or adding parameters.
• Q: How does a depthwise separable convolution save computation? A: It factorises spatial filtering (one kernel per channel) from channel mixing (1×1), reducing cost to 1/C_out + 1/k² of a standard conv — about 8–9× for 3×3 kernels.
• Q: Where do a VGG-16's parameters and its FLOPs sit? A: Roughly 90% of the ~138 M parameters are in the three fully connected layers, while most FLOPs are in the early high-resolution conv layers — which is why GAP heads cut size but not latency.
• Q: Your model is 99% accurate on your test split but fails on field images. First checks? A: Look for train/test leakage and distribution shift — camera, lighting, resolution, class balance; verify the split is by subject/site rather than random, and match preprocessing and augmentation to field conditions.
• Q: Why are CNNs still preferred over vision transformers on an edge device? A: Stronger local inductive bias means good accuracy from small datasets, and 3×3 convolutions map onto Winograd/INT8 kernels in TFLite, TensorRT and NPUs with lower memory bandwidth and latency.

Common mistakes

• Dropping the floor in the output-size formula, or forgetting that padding is added on both sides (hence +2P).
• Computing filter parameters as k×k and omitting C_in and the bias — a "3×3 filter" on a 64-channel input holds 576 weights, not 9.
• Assuming parameters grow with image size, or that a CNN trained at 224×224 can take any size when it has a flatten+dense head.
• Claiming pooling makes a CNN rotation- or scale-invariant; it only gives small-translation tolerance.
• Mixing up stride and dilation, or using "same" padding with an even kernel and expecting a symmetric result.
• Confusing parameter count with compute: reporting a model as "light" on parameters while its early high-resolution layers dominate latency.
• Forgetting that batch norm normalises per channel (not per neuron), that its running statistics must be frozen with model.eval() at inference, and that augmentations such as horizontal flip are invalid where orientation carries meaning.
• Applying softmax twice, or feeding already-softmaxed outputs into nn.CrossEntropyLoss, which expects raw logits.

Related topics

Neural networks and backpropagation · Training deep networks: optimisers, normalisation and regularisation · Transfer learning and fine-tuning · Transformers and attention · Generative models: GANs, VAEs and diffusion · Digital image processing and feature extraction