Network security: firewalls, VPN and IDS

In short

Network security is the set of controls that protect data in transit and control which hosts and services can reach each other: firewalls enforce a policy on traffic, VPNs create an authenticated and encrypted tunnel across an untrusted network, and intrusion detection/prevention systems watch traffic for attack patterns. It is the perimeter-and-segmentation layer of defence in depth, and the layer a student is most likely to be asked to configure in a first job. It carries dedicated marks in university Information Security / Cryptography and Network Security papers and in GATE CS/IT (network attacks, IPsec, firewalls), and dominates interviews for network and security engineer roles at TCS, Infosys, Wipro, HCL, Tech Mahindra, Cisco, Palo Alto Networks, Fortinet, Airtel, Jio, banks and the CERT-In empanelled audit firms. In Indian practice it is governed by the IT Act 2000, the CERT-In Directions of April 2022, RBI/SEBI cyber-security frameworks and ISO/IEC 27033.

Explained

Threat model: what network defences are defending against

• Reconnaissance: port and host scanning (TCP SYN/connect/FIN scans, UDP scans), banner grabbing, OS fingerprinting, DNS enumeration.
• Sniffing (passive): capturing cleartext traffic on a shared medium or mirrored port; defeated by encryption, not by firewalls.
• Spoofing and poisoning (active): IP spoofing, ARP cache poisoning, DNS cache poisoning/spoofing, MAC flooding to force a switch to behave like a hub, rogue DHCP.
• Man-in-the-middle and session hijacking: interception and relay of a session; TCP sequence-number prediction, cookie/session token theft.
• Availability attacks: SYN flood (half-open connections exhaust the backlog), UDP/ICMP floods, amplification/reflection (DNS, NTP, memcached), and distributed versions of all of them.
• Layer-wise view: L2 attacks need switch-port controls, L3/L4 attacks need firewalls and anti-spoofing, L7 attacks need proxies and application inspection. No single box covers all three.
• Defence in depth: perimeter firewall → DMZ → segmentation/VLANs → host firewall → IDS/IPS monitoring → logging. Each layer assumes the one outside it may fail.

Firewalls: types, working and architectures

• Definition: a firewall is a control point that permits or denies traffic between zones of different trust according to a rule base; it enforces the *access-control* part of network policy.
• Design principles: all traffic must pass through it; it should permit only explicitly allowed traffic (default deny); it must itself be hardened.
• 1. Packet-filtering (stateless) firewall: inspects each packet independently against the 5-tuple (source IP, destination IP, protocol, source port, destination port) plus flags and interface. Fast, cheap, protocol-blind; cannot tell a reply from an unsolicited packet, so return traffic needs risky permanent "high-port" rules.
• 2. Stateful inspection firewall: keeps a connection/state table and tracks the TCP state machine (and pseudo-state for UDP/ICMP); return traffic is allowed because the session exists. Handles fragmentation reassembly and, via ALGs, protocols that carry addresses in the payload (FTP active mode, SIP). State table size and timeout values are its scaling limits.
• 3. Application-level gateway (proxy) firewall: terminates the connection at L7, understands the protocol (HTTP, SMTP, DNS), can inspect and rewrite content, authenticate users and log URLs. Most secure, slowest, needs one proxy per protocol.
• 4. Circuit-level gateway: relays TCP connections at the session layer (SOCKS) without inspecting content; validates the handshake only.
• 5. NAT/PAT: not a security control by design, but hides internal addressing and implicitly blocks inbound connections to private addresses.
• 6. Next-generation firewall (NGFW): stateful inspection plus application identification independent of port (App-ID), user identity from AD/LDAP, integrated IPS, URL filtering, TLS/SSL decryption, threat-intelligence feeds and sandboxing. UTM is the same idea packaged for SMEs.
• 7. Host-based firewall: per-host rules (Windows Defender Firewall, iptables/nftables, firewalld, ufw); essential for east-west control. Cloud security groups are the same model, covered under cloud security.
• Architectures: *screening router* (single filtering router); *dual-homed/bastion host* (hardened, two interfaces, no routing); *screened host*; *screened subnet (DMZ)* with an outer and inner firewall and public servers in the middle; *three-legged firewall* (inside/outside/DMZ on one device) — the common Indian enterprise pattern.
• Rule base anatomy: rules are matched top-down, first match wins, ending with an implicit deny-all (make it explicit and logged). Order specific before general; put high-hit rules early for performance; log denies, not everything.
• Egress filtering: restrict outbound traffic too — it blocks C2 callbacks, data exfiltration and spoofed source addresses (BCP 38/RFC 2827 anti-spoofing: drop packets whose source address cannot legitimately arrive on that interface).
• Limitations: cannot inspect encrypted payloads without decryption; no protection against insiders, authorised-but-malicious traffic, application logic flaws, social engineering, or traffic that bypasses it (dongles, personal hotspots); misconfiguration and rule-base rot are the biggest real-world failure causes.

VPN: tunnelling, IPsec and TLS-based VPNs

• Definition: a virtual private network carries private traffic over a shared/public network using encapsulation (tunnelling) plus cryptographic protection, so the link behaves like a private leased line.
• Security services provided: confidentiality, integrity, peer authentication, anti-replay (sequence numbers), and traffic-flow confidentiality in tunnel mode. It does not by itself provide authorisation — a VPN user still needs firewall and NAC policy.
• Deployment types: remote access (client-to-site) for employees/WFH; site-to-site (gateway-to-gateway) between branch and HQ; extranet VPN for partners; provider-provisioned MPLS L3VPN/VPLS, which gives isolation but *no encryption* — a common exam trap.
• IPsec (RFC 4301 family), the examined standard:
  • AH (Authentication Header, IP protocol 51): integrity and authentication of the whole packet including the outer IP header; no confidentiality, and it breaks through NAT.
  • ESP (Encapsulating Security Payload, IP protocol 50): encryption plus optional integrity of the payload (not the outer header); the mode used in practice.
  • Transport mode: original IP header kept, only payload protected — host-to-host. Tunnel mode: whole original packet encapsulated in a new IP packet — gateway-to-gateway and remote access.
  • SA (Security Association): a one-way agreed set of parameters identified by SPI + destination address + protocol; two SAs are needed for bidirectional traffic. Stored in the SAD; the policy of what to protect is in the SPD.
  • IKE (UDP 500): Phase 1 authenticates peers and builds a secure channel via Diffie–Hellman (IKEv1 main/aggressive mode); Phase 2 (quick mode) creates the IPsec SAs. IKEv2 is the current choice — fewer messages, built-in NAT-T, EAP authentication, MOBIKE for roaming, reliable rekeying.
  • Supporting mechanisms: NAT-T encapsulates ESP in UDP 4500; DPD detects dead peers; SA lifetimes (time and bytes) force rekeying; PFS (fresh DH per rekey) ensures a stolen long-term key cannot decrypt past traffic.
• TLS/SSL VPN: runs over TCP/UDP 443, so it traverses hotel and café networks and strict proxies. Two forms: *clientless* (browser portal, reverse-proxies a few web apps) and *full-tunnel client* (OpenVPN, AnyConnect, GlobalProtect).
• Other/legacy protocols: PPTP (MS-CHAPv2 — considered broken, do not use), L2TP/IPsec, GRE (encapsulation only, no security), SSH port forwarding, and WireGuard (modern, small code base, UDP 51820, fixed modern cipher suite with ChaCha20-Poly1305, Curve25519 keys, no cipher negotiation).
• Split tunnelling vs full tunnelling: split sends only corporate subnets through the tunnel (saves bandwidth, risks a bridged compromised host); full tunnel sends everything, giving central inspection at the cost of gateway capacity. Add always-on and a kill switch for laptops.
• Practical pitfalls: MTU/fragmentation from tunnel overhead (fix with MSS clamping and PMTUD), overlapping private subnets between sites (needs NAT), DNS leakage, and gateway CPU limits on crypto throughput (use AES-NI/hardware offload).
• Evolution: ZTNA/SASE replaces "connect to the network" with per-application, continuously verified access brokered in the cloud — increasingly the model in Indian IT services for contractor and BYOD access.

IDS and IPS: detection, placement and accuracy

• IDS detects and alerts (passive, out-of-band); IPS sits inline and can drop, reset or rate-limit the traffic. An IPS is a control; an IDS is a sensor.
• By scope: NIDS (network sensor on a segment), HIDS (agent on a host: file integrity, logs, process and local-connection monitoring), WIDS (rogue AP/wireless), and NBAD (flow/NetFlow-based behaviour analysis).
• Placement: behind the perimeter firewall, in the DMZ, and on internal core links to see east-west traffic. Feed it via a SPAN/mirror port (can drop packets under load) or a passive network TAP (lossless). Inline devices need a fail-open vs fail-closed decision.
• Detection methods:
  • Signature/misuse detection: matches known patterns. Accurate for known attacks, cheap to explain, blind to zero-days; needs constant signature updates.
  • Anomaly detection: builds a statistical/ML baseline of normal and flags deviations. Can catch novel attacks but generates many false positives and must be retrained as "normal" changes; baselining a compromised network makes attacks look normal.
  • Stateful protocol analysis: compares traffic against the RFC-defined behaviour of the protocol.
  • Reputation/threat intel: blocks known-bad IPs, domains and file hashes.
• Outcome classes: TP (attack detected), FP (benign flagged — causes alert fatigue), FN (attack missed — the dangerous one), TN. Tuning trades FPR against FNR; the ROC curve plots TPR against FPR as the threshold moves.
• Evasion techniques to know: IP fragmentation and TCP segment overlap, encryption/tunnelling, polymorphic and encoded payloads, insertion/evasion using differing TTLs and checksums, protocol obfuscation, and low-and-slow or distributed timing to stay under thresholds.
• Tools: Snort and Suricata (signature NIDS/IPS), Zeek (protocol logging/behaviour), OSSEC/Wazuh (HIDS), Security Onion (bundled distro), Suricata/Zeek feeding a log platform. Alert triage and correlation belong to SOC/SIEM work.
• Deception: a honeypot is a decoy system with no production value — any traffic to it is suspicious, so it yields very few false positives; a honeynet is a network of them; tarpits slow scanners down. Low-interaction (emulated services) vs high-interaction (real OS, more learning, more risk).

Supporting network controls

• Segmentation: split the network by function and trust (user, server, OT/SCADA, card-data, management) so a breach cannot move laterally; microsegmentation does this per workload.
• VLANs and L2 hardening: separate broadcast domains, private VLANs, port security (MAC limits), DHCP snooping, dynamic ARP inspection, BPDU guard, disable unused ports and native-VLAN tagging. VLANs are isolation, not encryption.
• NAC / 802.1X: supplicant → authenticator (switch/AP) → RADIUS server with EAP; posture checks push non-compliant devices to a quarantine VLAN. Guest Wi-Fi stays on a separate SSID/VLAN with client isolation.
• Proxies and gateways: forward proxy with URL/content filtering for outbound; reverse proxy/load balancer terminating TLS for inbound; secure email and DNS filtering (RPZ, protective DNS).
• DDoS mitigation: SYN cookies, connection rate limits, blackholing, and upstream ISP or cloud scrubbing — volumetric attacks must be stopped before the last mile.
• Wireless: WPA2-Enterprise (802.1X) or WPA3 (SAE replaces the PSK handshake, resists offline dictionary attacks); never WEP or WPS.
• Zero trust principle: no implicit trust from network location; authenticate and authorise every connection and keep inspecting.

Indian standards, Acts and practice

• IT Act 2000 (with 2008 amendment): s.43 and s.66 (unauthorised access, damage, hacking), s.65, s.43A (reasonable security practices by body corporates), s.69/69B (interception and traffic-data monitoring powers), s.70B (CERT-In as national nodal agency), s.70A (NCIIPC for critical information infrastructure).
• CERT-In Directions of 28 April 2022 (in force from June 2022): report specified cyber incidents within 6 hours of noticing; maintain ICT system logs for 180 days within Indian jurisdiction; synchronise clocks to NIC/NPL NTP servers; designate a point of contact. Separately, *VPN service providers* (not enterprise/corporate VPNs, which the FAQs exempt) must keep subscriber records for five years — several commercial providers withdrew Indian servers in response. Verify the current text, as directions and FAQs are updated.
• Sector regulators: RBI cyber-security framework for banks (2016) and its baseline controls — network segmentation, firewall rule review, IDS/IPS, anti-APT, SOC; SEBI CSCRF for market intermediaries; IRDAI guidelines for insurers; DoT licence security conditions, MTCTE/ComSec testing of telecom equipment and the Trusted Telecom Portal for trusted sources.
• OSP (Other Service Provider) rules, liberalised 2020–21: BPO/ITES agents may work from home provided access is over a secure VPN to the authorised centre — the rule behind the WFH VPN build-outs at Indian BPOs.
• Standards and certification: ISO/IEC 27001 Annex A network controls and ISO/IEC 27033 (network security), NIST SP 800-41 (firewall policy) and SP 800-94 (IDPS), PCI DSS for card environments, Common Criteria/STQC evaluation, and CERT-In empanelled auditors for mandated audits. DPDP Act 2023 obliges data fiduciaries to take reasonable security safeguards and to report breaches — its rules were notified in 2025, so check current timelines.

Key points

• Default deny - permit only what is explicitly required; everything else is dropped and logged.
• Stateless vs stateful - a packet filter judges each packet alone; a stateful firewall tracks connections and allows return traffic automatically.
• Proxy firewall - terminates and understands the application protocol at L7; most thorough, least scalable.
• NGFW - adds application identity, user identity, IPS and TLS inspection to stateful inspection.
• DMZ / screened subnet - a buffer zone holding public-facing servers so a compromise does not reach the internal LAN.
• First-match-wins - rule order changes behaviour; a broad allow above a specific deny silently defeats it.
• Egress filtering and BCP 38 - control outbound traffic and drop spoofed source addresses.
• AH vs ESP - AH authenticates (including the outer header) with no encryption and breaks NAT; ESP encrypts and is what is deployed.
• Transport vs tunnel mode - transport protects the payload between hosts; tunnel wraps the whole packet between gateways.
• SA and SPI - an IPsec SA is unidirectional and identified by its SPI; two are needed per bidirectional session.
• IKEv2 - current key-exchange protocol: DH for keys, mutual authentication, NAT-T, MOBIKE, rekeying with PFS.
• IPsec vs SSL VPN - IPsec gives full network-layer access from managed devices; TLS VPN on 443 traverses restrictive networks and suits unmanaged devices.
• Split tunnel - saves bandwidth but removes central inspection of the user's other traffic.
• IDS vs IPS - IDS alerts out-of-band (SPAN/TAP); IPS is inline and blocks, so a false positive becomes an outage.
• Signature vs anomaly - signatures catch known attacks precisely; anomaly detection can catch novel attacks with far more false positives.
• Base rate fallacy - with rare attacks, even a tiny false-positive rate makes most alerts false, so precision matters more than accuracy.
• Honeypot - a decoy with no production role; traffic to it is inherently suspicious.
• CERT-In 2022 directions - 6-hour incident reporting, 180-day log retention in India, NIC/NPL time sync.

Formulas and examples

Example 1 - IDS alert quality (base rate fallacy). An NIDS inspects 10,00,000 (1 million) sessions a day. Attacks are 0.01% of sessions (p = 0.0001). Sensor: TPR = 99%, FPR = 0.5%.

• Attacks = 1,000,000 × 0.0001 = 100; benign = 999,900.
• TP = 0.99 × 100 = 99; FN = 1; FP = 0.005 × 999,900 = 4,999.5 ≈ 5,000; TN = 999,900 − 4,999.5 = 994,900.5.
• Precision = 99 / (99 + 4,999.5) = 99 / 5,098.5 = 1.94% — about 1 useful alert in 51.
• Accuracy = (99 + 994,900.5)/1,000,000 = 99.5% — high, yet the sensor is almost useless for triage: accuracy is the wrong metric for rare events.
• Tune FPR down to 0.01%: FP = 999,900 × 0.0001 = 99.99 ≈ 100, precision = 99/199 = 49.7%. Cutting FPR 50× raised usable alerts from 1.9% to ~50%.

Example 2 - IPsec MTU and MSS clamping. Ethernet path MTU 1500 B; ESP tunnel mode, AES-128-CBC + HMAC-SHA1-96, IPv4.

• Fixed overhead = 20 + 8 + 16 + 2 (pad-length + next-header) + 12 = 58 B, plus padding to a 16-byte boundary.
• Largest inner packet: encrypted block ≤ 1500 − 20 − 8 − 16 − 12 = 1,444 B; the largest multiple of 16 is 1,440, so inner packet ≤ 1,440 − 2 = 1,438 B.
• Using a safe inner MTU of 1,420 B: block = 1,420 + 2 = 1,422 → padded to 1,424; total = 20 + 8 + 16 + 1,424 + 12 = 1,480 B ≤ 1500 ✔ (no fragmentation).
• MSS to clamp = 1,420 − 20 − 20 = 1,380 B — the familiar 1,380 value on IPsec tunnels.

Example 3 - Sizing a site-to-site VPN link. 200 concurrent users average 300 kbps each across a 100 Mbps leased line, over the tunnel of Example 2.

• Payload = 200 × 300 kbps = 60,000 kbps = 60 Mbps.
• Overhead factor = 1,480 / 1,420 = 1.0423 (4.23%).
• On-wire = 60 × 1.0423 = 62.5 Mbps → 62.5% utilisation; acceptable (keep ≤70–80%).
• At 300 users: 90 × 1.0423 = 93.8 Mbps = 93.8% → the link must be upgraded or split tunnelling used.

Example 4 - Firewall rating vs packet size (small-packet DoS). A firewall is rated "1 Gbps throughput, 5,00,000 pps".

• At 512-byte average packets: pps needed = 1×10⁹ / (512 × 8) = 1×10⁹/4,096 = 2,44,141 pps → within the 500,000 pps limit ✔
• At 64-byte packets: pps needed = 1×10⁹/512 = 19,53,125 pps → far beyond the limit.
• Actual throughput at 64 B = 500,000 × 64 × 8 = 256 Mbps. A flood of tiny packets collapses it to a quarter of its headline rating — always read vendor figures with the test packet size.

Example 5 - HA firewall availability. Each firewall: MTBF = 50,000 h, MTTR = 4 h.

• A = 50,000 / 50,004 = 0.99992 (99.992%); U = 8.0 × 10⁻⁵ → annual downtime = 8.0×10⁻⁵ × 8,760 = 0.70 h ≈ 42 min.
• Active–passive pair, independent failures: U_pair = (8.0×10⁻⁵)² = 6.4×10⁻⁹ → downtime ≈ 6.4×10⁻⁹ × 8,760 h = 0.2 s/year.
• Real clusters do far worse because failures are correlated — shared power, a bad firmware upgrade or one wrong rule pushed to both nodes.

# Stateful iptables skeleton: default deny, allow established, SSH from LAN only
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -s 192.168.10.0/24 -p tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -m conntrack --ctstate NEW -j ACCEPT
iptables -A INPUT -j LOG --log-prefix "FW-DROP: "

# Snort/Suricata rule anatomy: action proto src port -> dst port (options)
alert tcp any any -> 10.0.0.0/24 22 (msg:"SSH brute force"; flow:to_server;
  threshold:type threshold, track by_src, count 10, seconds 60; sid:1000101; rev:1;)

Where it is used

• IT services and managed security providers (TCS, Infosys, Wipro, HCLTech, Tech Mahindra, LTIMindtree): firewall rule management, VPN provisioning, IDS/IPS tuning for global clients; roles such as network security engineer and L1/L2 security analyst.
• Banks, NBFCs and payment companies: RBI-mandated segmentation, DMZ design, IPsec links to RBI/NPCI, PCI DSS firewall-rule reviews every six months; UPI and core-banking networks.
• Telecom and ISPs (Airtel, Jio, BSNL, Vodafone Idea): carrier-grade firewalls, DDoS scrubbing centres, MPLS VPN products for enterprises, lawful-interception compliance under DoT licence conditions.
• Product and OEM engineering: Cisco, Palo Alto Networks, Fortinet, Check Point, Juniper, Sophos, SonicWall and Indian firms such as Seqrite/Quick Heal and Haltdos build and support these devices from India.
• Government, PSUs and critical infrastructure: NIC/NICNET VPNs for state departments, NCIIPC-guided protection of power, railways and refineries, air-gapping and one-way gateways for OT/SCADA.
• Campus and enterprise networks: 802.1X on institute Wi-Fi, VLANs for hostels/labs/admin, remote-access VPN for faculty into licensed journals and lab resources.

Interview questions

• Q: What is the difference between a stateless and a stateful firewall? A: A stateless filter evaluates every packet independently against the 5-tuple; a stateful firewall maintains a connection table and the TCP state, so it allows return traffic for established sessions and can block out-of-state packets.
• Q: Why is rule order important in a firewall? A: Rules are evaluated top-down and the first match is applied, so a broad allow placed above a specific deny silently overrides it; the implicit deny-all sits at the bottom.
• Q: What is a DMZ and why place public servers there? A: A DMZ is a separately firewalled zone between the internet and the LAN; if a public web or mail server is compromised, the attacker is still separated from internal systems by a second rule set.
• Q: AH or ESP — which would you deploy and why? A: ESP, because it provides confidentiality as well as integrity and works through NAT; AH authenticates the outer IP header too, which NAT rewrites, so it breaks.
• Q: Difference between IPsec transport and tunnel mode? A: Transport mode protects only the payload and keeps the original IP header (host-to-host); tunnel mode encapsulates the entire original packet in a new IP packet (gateway-to-gateway, remote access), also hiding the internal addresses.
• Q: What does an IPsec SA contain and how many are needed? A: Keys, algorithms, mode, lifetime and sequence counters, identified by the SPI plus destination and protocol; SAs are unidirectional, so a two-way session needs two.
• Q: When would you choose an SSL VPN over IPsec? A: For unmanaged or BYOD devices and users on restrictive networks — it runs on TCP 443, needs little or no client, and can be scoped to specific applications rather than the whole subnet.
• Q: Why do users report broken websites after a VPN is enabled? A: Tunnel overhead reduces the effective MTU; packets with DF set are dropped and PMTUD is often blocked by filtered ICMP, so you clamp the TCP MSS (typically to ~1,380) on the tunnel interface.
• Q: IDS versus IPS — which is riskier to deploy, and why? A: The IPS, because it is inline: a false positive drops legitimate traffic and a device failure can break the path, so it needs careful tuning and a fail-open/fail-closed decision.
• Q: Signature-based or anomaly-based detection? A: Signature detection is precise for known attacks but blind to new ones; anomaly detection can spot novel behaviour but produces many false positives and needs clean baselining — production sensors use both.
• Q: My IDS is 99% accurate — is it good? A: Not necessarily; with attacks at 0.01% prevalence, a 0.5% false-positive rate makes about 98% of alerts false. Judge a sensor on precision and recall, not accuracy.
• Q: A firewall and an IDS are in place — can an attacker still get in? A: Yes: over permitted ports and encrypted channels, via stolen VPN credentials, through an insider or a compromised endpoint, or by exploiting an application flaw the firewall is allowed to pass.

Common mistakes

• Writing a rule base that ends in an implicit allow, or leaving any-any test rules at the top — and never reviewing or removing stale rules.
• Claiming a firewall stops viruses, phishing or SQL injection; a packet/stateful filter sees headers, not application logic.
• Saying AH encrypts, or that AH and ESP can be used interchangeably behind NAT.
• Confusing tunnel and transport mode, or treating MPLS VPN and VLAN isolation as encryption.
• Deploying an IPS inline in blocking mode without a learning/alert-only period, then causing a production outage on a false positive.
• Judging an IDS by accuracy instead of precision/recall, and ignoring the base-rate effect on alert volume.
• Hanging the VPN pool directly in the internal LAN with no firewall or NAC policy, so a VPN login equals full internal access.
• Ignoring Indian compliance detail — 6-hour CERT-In reporting, 180-day log retention within India and NTP synchronisation — when designing logging for a firewall or IDS.

Related topics

Information security fundamentals · Cryptography basics · Security operations: SOC and SIEM · Ethical hacking and penetration testing · Cloud security · Computer networks: TCP/IP protocol stack