learn deep. ship safe.Practice testsFollow ByteMunSubscribe

Concept

Streams

Processing data piece by piece instead of loading it all into memory, with backpressure.

Streams explained

Streams in Node.js process data piece by piece as it arrives instead of loading it all into memory first. Reading a large file, handling an HTTP upload or piping a database export to a client can then use a small, constant amount of memory. Node has four kinds: Readable, Writable, Duplex (both) and Transform (changes data as it passes through).

How streams work

A readable stream emits chunks (Buffers or strings); a writable stream accepts them. pipeline connects them, passes errors along and closes everything when done.

import { createReadStream, createWriteStream } from "node:fs";
import { createGzip } from "node:zlib";
import { pipeline } from "node:stream/promises";

// Compress a large file using little memory, whatever its size
await pipeline(
  createReadStream("access.log"),
  createGzip(),                       // a Transform stream
  createWriteStream("access.log.gz"),
);

Backpressure

If the reader is faster than the writer, data piles up in memory. Backpressure is the mechanism that slows the reader: write() returns false when the writer's buffer is full, and the reader pauses until the drain event. pipeline and pipe handle this for you; writing manually in a loop and ignoring the return value is how memory blows up.

Where streams are used

  • HTTP requests and responses are streams: you can stream a file to a client without reading it first.
  • File processing: logs, CSV exports, compression.
  • Async iteration: for await (const chunk of stream) reads a stream with ordinary loop syntax.

Common mistakes

  • Reading a whole file with readFile when it can be gigabytes.
  • Chaining .pipe() without handling errors on every stream; pipeline fixes this.
  • Ignoring backpressure in custom write loops.

What interviewers ask about streams

  • "Why use streams?" Constant memory and starting to process before all data has arrived.
  • "What is backpressure?" Slowing a fast producer to match a slow consumer.
  • "pipe or pipeline?" pipeline forwards errors and cleans up.

Streams in web responses

A server can stream a response as it is produced: a large CSV export, a file download or an AI model's tokens as they are generated. The client starts receiving data immediately, memory stays flat on the server, and the response finishes when the stream ends.

Concept map
Part of Node.jsStreams

What should you learn before Streams?

  1. FunctionsReusable blocks of code; in JavaScript they are values that can be passed around and returned.
  2. CallbacksFunctions passed to other functions to be called later, the original way to handle async work.
  3. BuffersFixed-size chunks of raw binary data in Node.js.
  4. The event loopHow a single-threaded runtime runs callbacks from the task and microtask queues.

Connections to other concepts

Part of
Node.js

Practise Streams

All topics