Streams API in Practice: Streaming Large Files in the Browser

Why Stream Processing?

Traditional large-file reads:

// ❌ 200MB file → 200MB ArrayBuffer → possible OOM
const buffer = await file.arrayBuffer();
processEntireBuffer(buffer);

Streaming:

// ✅ Process 64KB at a time
const stream = file.stream();
for await (const chunk of stream) {
  processChunk(chunk); // 64KB
}

Compare	Full load	Streaming
Memory	File size	~64KB (chunk size)
Start processing	After full read	First chunk arrives
200MB file	200MB RAM	~64KB RAM
1GB file	May OOM	Runs fine

---

Three Core Stream Types

ReadableStream  →  source (file, network response)
TransformStream →  middle (compress, encrypt, convert)
WritableStream  →  sink (download, upload)

ReadableStream

const response = await fetch('/large-file.pdf');
const reader = response.body!.getReader();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  console.log(`Received ${value.byteLength} bytes`);
}

TransformStream

const compressor = new TransformStream({
  transform(chunk, controller) {
    const compressed = gzipSync(chunk);
    controller.enqueue(compressed);
  }
});

// pipe
fileStream.pipeThrough(compressor).pipeTo(downloadStream);

---

Backpressure

When the producer is faster than the consumer, the stream pauses the producer:

FileReader (fast) → [buffer full] → pause read
                      ↓
              TransformStream (slow) → consume chunk → space free → resume read

That’s the main advantage over manual chunk loops—automatic flow control without hand-tuned buffer sizes.

---

Streaming in ToolsKu

Gzip compress/decompress

The compress tool uses CompressionStream:

const compressed = file.stream().pipeThrough(new CompressionStream('gzip'));
// stream compression without loading the whole file

Large PDF chunked reads

async function* readPdfChunks(file: File, chunkSize = 4 * 1024 * 1024) {
  let offset = 0;
  while (offset < file.size) {
    const chunk = file.slice(offset, offset + chunkSize);
    yield await chunk.arrayBuffer();
    offset += chunkSize;
  }
}

for await (const chunk of readPdfChunks(largePdf)) {
  await processPdfChunk(chunk);
}

Streaming download

const stream = new ReadableStream({
  async start(controller) {
    for await (const chunk of generateChunks()) {
      controller.enqueue(chunk);
    }
    controller.close();
  }
});

const response = new Response(stream, {
  headers: { 'Content-Disposition': 'attachment; filename="output.zip"' }
});

---

Stream API Compatibility

API	Chrome	Firefox	Safari	Edge
ReadableStream	52+	65+	10.1+	79+
TransformStream	80+	102+	14.1+	80+
CompressionStream	80+	113+	16.4+	80+
File.stream()	76+	69+	14.1+	79+

CompressionStream is newer; unsupported browsers fall back to full-buffer processing with pako/fflate.

---

Summary

The Streams API is the modern standard for large files. ReadableStream for chunked reads, TransformStream for middle processing, backpressure for flow control—together they make GB-scale browser processing feasible and underpin ToolsKu’s large-file tools (PDF, video, compression).