Back-end development with Node.js

In short

Node.js is a server-side JavaScript runtime built on Google's V8 engine plus the C library libuv, which gives it a single-threaded event loop with non-blocking I/O. Because one thread can hold thousands of open sockets while waiting on the database or network, Node suits I/O-heavy back-ends — APIs, gateways, chat, streaming, notification services — but is a poor fit for long CPU-bound work. It is the standard back-end of the MERN stack taught in Indian university electives, AICTE "Full Stack Development" courses and almost every campus training programme, and it is asked in interviews at Flipkart, Swiggy, Zomato, Paytm, PhonePe, Razorpay, Jio, Ola, Dream11 and at every service company staffing MERN projects (TCS, Infosys, Wipro, Accenture, Cognizant). University exams test the event loop, modules, streams and Express; placement rounds test async ordering, error handling and scaling.

Explained

Architecture: runtime, event loop and the thread pool

• Node.js is not a language and not a framework — it is a runtime: V8 (compiles and runs JavaScript), libuv (event loop, thread pool, async file and network I/O), plus C++ bindings and a standard library (fs, http, path, crypto, os, stream, events, net, url, zlib, child_process, worker_threads).
• Single-threaded event loop, multi-threaded internals. Your JavaScript runs on one thread. Network I/O is handed to the OS (epoll on Linux, IOCP on Windows); file system, DNS (dns.lookup), zlib and most crypto work goes to libuv's thread pool (UV_THREADPOOL_SIZE, default 4).
• Event-driven, non-blocking model: a request registers a callback and the thread moves on; when the kernel reports the data is ready, the callback is queued and run. No thread is parked per request — unlike the classic thread-per-request model of Apache or a default Java servlet container.
• Phases of one event-loop iteration (tick), in order:
  1. timers — due setTimeout / setInterval callbacks;
  2. pending callbacks — some deferred system callbacks (e.g. TCP errors);
  3. idle, prepare — internal;
  4. poll — wait for and execute I/O events (where the loop usually blocks);
  5. check — setImmediate callbacks;
  6. close callbacks — e.g. socket.on('close').
• Microtasks run between phases, after each callback: the process.nextTick queue drains first, then the promise microtask queue. So nextTick beats Promise.then, and both beat any timer or setImmediate.
• Blocking the loop blocks everyone. A 300 ms synchronous loop, JSON.parse of a huge body, fs.readFileSync, crypto.pbkdf2Sync or a regex with catastrophic backtracking stalls every concurrent request on that process.

Modules, npm and project structure

• CommonJS — require() / module.exports; synchronous, cached after first load, the historical Node default (.cjs, or .js without "type": "module").
• ES Modules (ESM) — import / export; static, asynchronous, supports top-level await; enabled by "type": "module" in package.json or a .mjs extension. require of ESM is now possible in recent Node versions, but mixed projects still break on __dirname, which ESM replaces with import.meta.url.
• Module resolution order: core module → relative/absolute path → node_modules upward through parent directories.
• package.json fields: name, version, main/exports, type, scripts, dependencies, devDependencies, peerDependencies, engines.
• Semantic versioning (MAJOR.MINOR.PATCH) with range operators: ^4.18.2 allows any 4.x below 5.0.0; ~4.18.2 allows only 4.18.x; a bare 4.18.2 pins exactly.
• **package-lock.json** freezes the whole resolved tree — commit it. npm ci installs strictly from the lock file (use it in CI and on servers); npm install may update it.
• Globals that are not browser globals: process (process.env, argv, exit, on('SIGTERM')), __dirname/__filename (CJS), Buffer, global/globalThis. There is no window or document.
• Configuration never goes in code — read it from process.env (loaded from a .env file that is git-ignored) so the same build runs on dev, staging and production.

Asynchronous programming: callbacks, promises, async/await

• Error-first callbacks — fn(err, data); nesting them produces callback hell (deep pyramids, duplicated error handling).
• Promises — a pending/fulfilled/rejected object; chained with .then/.catch/.finally. util.promisify wraps old callback APIs; fs/promises, dns/promises and timers/promises are built in.
• async/await — syntactic sugar over promises; await suspends only that async function, not the event loop. Errors are caught with try/catch.
• Combinators: Promise.all (fails fast, all must succeed), Promise.allSettled (every outcome returned), Promise.race (first settled — used for timeouts), Promise.any (first fulfilled).
• Sequential vs parallel: three awaited calls of 100 ms each in sequence take ~300 ms; await Promise.all([...]) of the same three takes ~100 ms. A frequent interview trap.
• EventEmitter — the publish/subscribe backbone (on, once, emit, off). Streams, http.Server and process are all emitters. More than 10 listeners on one event triggers a memory-leak warning.
• Unhandled rejections terminate the process by default in current Node versions — always attach a .catch or wrap in try/catch.

Streams, buffers and the file system

• Buffer — a fixed-length raw byte array outside the V8 heap, for binary data; encodings utf8, base64, hex, ascii. Never use the deprecated new Buffer(); use Buffer.from / Buffer.alloc.
• Four stream types: Readable (fs.createReadStream, HTTP request), Writable (HTTP response, file write), Duplex (TCP socket), Transform (zlib.createGzip, a CSV parser).
• Why streams matter: they process data in chunks (highWaterMark, 64 KB default for file streams), so memory stays constant and the first byte reaches the client sooner. Reading a 2 GB file into a buffer instead will exhaust memory.
• Backpressure: writable.write() returning false means the consumer is slower than the producer; pipeline() (from stream/promises) handles backpressure, errors and cleanup — prefer it to raw .pipe().
• **fs has three flavours:** callback (fs.readFile), promise (fs/promises), and synchronous (fs.readFileSync — acceptable only at start-up, in CLI scripts, never inside a request handler).

Express.js and the HTTP layer

• **Core http module:** http.createServer((req, res) => ...) with req a Readable stream and res a Writable stream. Express, Fastify, Koa, NestJS and Hapi are built on it; Express is the one examined.
• Middleware is a function (req, res, next) executed in registration order: logging, express.json() body parsing, CORS, compression, authentication, validation, then the route handler. Calling next() passes control; not calling it (and not responding) hangs the request.
• Routing: app.get/post/put/patch/delete, route parameters (/jobs/:id → req.params.id), query (req.query), body (req.body), express.Router() for modular route files.
• Error-handling middleware has four arguments (err, req, res, next) and must be registered last. In Express 4, errors thrown inside an async handler are not caught automatically — wrap the handler or use a catchAsync helper (Express 5 forwards rejected promises).
• Layered project structure: routes → controllers → services → repositories/models, with config, middlewares, utils and tests alongside — keeps business logic testable without HTTP.
• Database access: pg/mysql2 with a connection pool (never one connection per request), Mongoose or the MongoDB driver, Prisma, Sequelize or TypeORM as ORMs. Always parameterise queries; string concatenation is how SQL injection happens.

Scaling, performance and production operation

• **cluster** forks one worker process per CPU core, all sharing a listening socket (round-robin on Linux by default) — the standard way to use a multi-core VM. State cannot be shared between workers, so sessions/caches must move to Redis or the database.
• **worker_threads** runs JavaScript on real OS threads inside one process, sharing memory via SharedArrayBuffer — for CPU-bound work (image resizing, parsing, hashing) without forking.
• **child_process** (spawn, exec, fork) runs external commands or another Node script; spawn streams output and is safe for large output, exec buffers it.
• Process managers: PM2 or systemd restart crashed processes, run cluster mode and rotate logs; a reverse proxy (Nginx) terminates TLS, serves static files and load-balances.
• Graceful shutdown: on SIGTERM, stop accepting new connections, finish in-flight requests, close database pools, then exit — essential behind a load balancer or an orchestrator.
• Observability: structured JSON logs (pino, winston), health endpoints, metrics, --inspect with Chrome DevTools, --prof/clinic for CPU profiles, heap snapshots for leaks.
• Security practice: helmet for headers, rate limiting, input validation (zod, joi), npm audit, least-privilege DB users, secrets in environment variables or a vault, HTTPS only. Indian compliance that affects design: the Digital Personal Data Protection Act, 2023 (consent, purpose limitation, breach notification to the Data Protection Board - duties from 13 May 2027; until then IT Act s.43A and the SPDI Rules 2011), RBI's payment-data localisation directive (payment system data stored in India), and CERT-In's 2022 directions (180-day log retention in India and 6-hour incident reporting — verify the current text before relying on it).
• Testing: the built-in node:test runner and node:assert, or Jest/Mocha+Chai; supertest for HTTP routes; nock or msw to stub outbound calls.
• Version policy: up to Node 26 (April 2026, LTS from October 2026), a new major shipped every six months and only even-numbered majors became LTS (30 months of LTS support). From Node 27 (April 2027) there is one major a year and every major becomes LTS. Build on an LTS line and confirm the current LTS version before pinning, since the lines advance each year.

Key points

• Node.js - a V8 + libuv runtime for server-side JavaScript, event-driven and non-blocking.
• Event loop - single JavaScript thread cycling through timers → pending → poll → check → close phases.
• libuv thread pool - default 4 threads serving fs, DNS, zlib and crypto; set by UV_THREADPOOL_SIZE.
• Non-blocking I/O - the thread registers a callback instead of waiting, so one process holds thousands of sockets.
• process.nextTick vs Promise vs setImmediate vs setTimeout - nextTick queue first, then promises, then timers, then check phase.
• Blocking code - any synchronous CPU work freezes all concurrent requests in that process.
• CommonJS vs ESM - require/module.exports (sync, cached) versus import/export (static, async, top-level await).
• package-lock.json - exact resolved dependency tree; npm ci installs from it reproducibly.
• Semver ranges - ^ allows minor+patch upgrades, ~ allows patch only, bare version pins.
• EventEmitter - on/emit pub-sub used by streams, servers and process.
• Streams - Readable, Writable, Duplex, Transform; chunked processing with constant memory and backpressure.
• Buffer - raw bytes stored off the V8 heap; created with Buffer.from/Buffer.alloc.
• Middleware - ordered (req, res, next) functions; the error handler takes four arguments and comes last.
• cluster vs worker_threads - separate processes (no shared memory) versus threads inside one process (shared memory) for CPU work.
• Connection pooling - reuse a fixed set of DB connections; one per request exhausts the database.
• Graceful shutdown - handle SIGTERM, drain in-flight requests, close resources, exit.
• Best fit - I/O-bound APIs, gateways, real-time apps; worst fit - heavy computation, video encoding, large matrix work.
• Node is not multi-threaded JavaScript - scale out with processes/threads, never by hoping the loop copes.

Formulas and examples

Example 1 — concurrency from Little's Law. An Express job-search API serves X = 1,200 req/s with an average response time R = 150 ms, almost all of it waiting on PostgreSQL.

• N = X × R = 1,200 × 0.150 = 180 requests in flight on one event loop.
• If each in-flight request holds about 40 KB of buffers, that is 180 × 40 = 7,200 KB ≈ 7.0 MB — trivial, which is exactly why Node handles it on one thread.
• But the DB pool must be sized for the *database* work: if DB time is 120 ms of the 150 ms, in-flight DB calls = 1,200 × 0.120 = 144 — so a 10-connection pool queues badly and a sharded/replica design or caching is needed.

Example 2 — a blocking handler saturates the loop. A route calls crypto.pbkdf2Sync costing S_cpu = 25 ms, with arrivals of 120 req/s.

• One thread's capacity X_max = 1/0.025 = 40 req/s.
• U = X × S = 120 × 0.025 = 3.0 (300%) → the queue grows without bound; latency climbs until timeouts.
• Fix A: cluster on an 8-vCPU box with k = 8 → X_cluster at U_target 0.7 = 8 × 40 × 0.7 = 224 req/s, U = 120/320 = 0.375.
• Fix B: use the async crypto.pbkdf2 so the work goes to the thread pool, or move hashing into worker_threads.

Example 3 — thread-pool ceiling. Each request does one async pbkdf2 of T = 100 ms with the default P = 4.

• X_pool = P/T = 4/0.1 = 40 req/s, regardless of how fast V8 is — requests 41+ queue inside libuv.
• Raising UV_THREADPOOL_SIZE=8 on an 8-vCPU machine gives 8/0.1 = 80 req/s; beyond the core count the threads only contend for CPU.

Example 4 — streaming a large file. A 1.5 GB (1,536 MB) CSV of listings must be transformed and written out.

• fs.readFile needs ≈ 1,536 MB resident → exceeds a typical container limit and the V8 string/buffer limits.
• createReadStream with highWaterMark 64 KB and 20 concurrent streams ≈ 20 × 64 KB = 1,280 KB ≈ 1.25 MB of buffers.
• At a disk throughput of 120 MB/s, time = 1,536 / 120 = 12.8 s, and the first row is available in milliseconds.

Example 5 — heap sizing for an in-process cache. A worker is started with --max-old-space-size=512 (512 MB old space) and caches 300,000 entries of ~1.6 KB each.

• Cache size = 300,000 × 1.6 KB = 480,000 KB = 468.8 MB — plus the working set this exceeds 512 MB → long GC pauses then JavaScript heap out of memory.
• Halving to 150,000 entries gives 240,000 KB = 234.4 MB, leaving headroom; the real fix for a 4-worker cluster is one shared Redis cache instead of 4 × 234 MB of duplicated data.

Example 6 — async ordering (classic placement question).

console.log('A');
setTimeout(() => console.log('B'), 0);
setImmediate(() => console.log('C'));
process.nextTick(() => console.log('D'));
Promise.resolve().then(() => console.log('E'));
console.log('F');

Output: A, F, D, E, B, C — synchronous lines first, then the nextTick queue, then promise microtasks, then the timers phase, then the check phase. (B before C is the usual result in the main module but is not guaranteed; inside an I/O callback setImmediate always runs before a 0 ms timer.)

Where it is used

• Consumer internet back-ends and BFF layers: Swiggy, Zomato, Flipkart, Myntra, Ola and Dream11 use Node services as API gateways/backend-for-frontend aggregating many microservices for mobile and web apps.
• Fintech dashboards and integration services: Razorpay, Paytm and PhonePe style merchant dashboards, webhook receivers and UPI/payment-gateway callback handlers — I/O-bound, latency-sensitive, and bound by RBI localisation and the DPDP Act, 2023.
• Real-time features: chat, live scores, driver/delivery tracking, auction and ticketing systems using WebSockets (ws, Socket.IO) with Redis for fan-out.
• Edtech, SaaS and internal tools: Byju's/Unacademy-type platforms, HRMS and admin portals; also serverless functions on AWS Lambda where Node has the fastest cold starts.
• Services companies and campus hiring: "MERN full-stack developer", "Node.js backend developer", "SDE-1 (Node)" roles at TCS, Infosys, Wipro, HCLTech, Cognizant, Accenture and thousands of startups; typical interview stack is Node + Express + MongoDB/PostgreSQL + Docker + AWS.
• Tooling: build tools, CLIs, scrapers, cron jobs and data pipelines written in Node because the team already knows JavaScript.

Interview questions

• Q: Node.js is single-threaded — how does it handle thousands of concurrent requests? A: JavaScript runs on one thread, but I/O is delegated to the OS and to libuv's thread pool; the loop only runs callbacks when data is ready, so no thread waits per request.
• Q: What exactly blocks the event loop? A: Any synchronous CPU work in JavaScript — tight loops, readFileSync, JSON.parse of a huge payload, sync crypto, catastrophic regex — because the single thread cannot run other callbacks meanwhile.
• Q: Difference between process.nextTick and setImmediate? A: nextTick callbacks run immediately after the current operation, before the loop continues; setImmediate runs in the check phase of the next iteration, after poll — so nextTick can starve the loop if used recursively.
• Q: setTimeout(fn, 0) vs setImmediate(fn)? A: Inside an I/O callback setImmediate always fires first (check phase precedes the next timers phase); in the main module the order is not guaranteed because the first timers phase may already be due.
• Q: Why are streams preferred over reading a whole file? A: Streams process fixed-size chunks, so memory stays constant regardless of file size and the first bytes reach the client earlier; a full read scales memory with file size and can crash the process.
• Q: What is backpressure and how do you handle it? A: It is the signal (write() returning false) that the writable side is slower than the readable side; handle it by waiting for drain or by using pipeline(), which manages it and cleans up on error.
• Q: How do you use all 8 cores of a server? A: Fork 8 workers with cluster (or run 8 containers behind a load balancer) and move shared state to Redis or the database, since worker processes do not share memory.
• Q: cluster or worker_threads for image resizing? A: worker_threads — it is CPU-bound work that can be offloaded to a thread inside the same process with cheap start-up and shared memory, keeping the main loop responsive.
• Q: Why doesn't Express 4 catch an error thrown inside an async route handler? A: The thrown value becomes a rejected promise that Express 4 never awaits, so next(err) is never called; wrap handlers in a helper that catches and forwards, or use Express 5.
• Q: How does error-handling middleware differ from normal middleware? A: It takes four parameters (err, req, res, next), is registered last, and runs only when next(err) is called or an error is forwarded.
• Q: CommonJS vs ES Modules? A: CommonJS uses synchronous cached require/module.exports; ESM uses static import/export with asynchronous loading and top-level await, enabled by "type": "module" or .mjs, and has no __dirname.
• Q: Why commit package-lock.json and use npm ci on servers? A: The lock file pins every transitive version so builds are reproducible; npm ci installs exactly that tree and fails if package.json and the lock disagree, whereas npm install may silently upgrade.
• Q: What happens on an unhandled promise rejection? A: In current Node versions the process prints the error and exits with a non-zero code by default, so every async path needs a .catch or try/catch plus a process-level handler for logging.

Common mistakes

• Saying "Node is single-threaded" without mentioning libuv's thread pool and the OS — or claiming Node cannot use multiple cores at all.
• Using fs.readFileSync, sync crypto or a long for loop inside a request handler, then wondering why p99 latency explodes under load.
• Awaiting independent calls one after another instead of Promise.all, tripling response time.
• Forgetting the four-argument error middleware, or registering it before the routes so it never fires.
• Opening a new database connection per request instead of using a pool, and never closing it on error.
• Storing sessions, counters or caches in process memory and then enabling cluster/multiple containers — state silently diverges per worker.
• Assuming setTimeout(fn, 0) runs before setImmediate everywhere, or that process.nextTick is "the same as" a microtask promise.
• Committing .env secrets, hardcoding ports/credentials, or skipping input validation and parameterised queries.
• Reporting a load test from a single-worker dev machine as production capacity, without stating cores, worker count and concurrency.

Related topics

REST APIs · Authentication and authorisation: sessions, OAuth and JWT · Caching, queues and CDNs · Containers: Docker and Kubernetes · Front-end frameworks: React basics · System design basics for freshers