WebRTC in Production: Signaling, NAT Traversal, SFU Architecture, and Real-World Pitfalls

网络协议

The Call That Dropped 40% of Users

"Video calls are failing." The dashboard showed 40% connection failure. Users on mobile networks, behind corporate firewalls, in regions with CGNAT — all unable to connect. Our single TURN server had hit its limit: 10,000 concurrent relay connections at 2 Mbps each, pushing 20 Gbps. The kernel was silently dropping UDP packets because the receive buffer was full. We learned that WebRTC demos work on localhost, but production demands a completely different level of understanding.

I've deployed four production WebRTC systems — from a telehealth platform handling 5,000 concurrent consultations to a live streaming service with 50,000 viewers. Here's everything I wish I had known from day one.

Core Concepts Reference

Concept Role Key Mechanism Production Impact
Signaling Exchange session metadata before media WebSocket/SSE/HTTP long-poll Reconnection, message ordering, room state
ICE Interactive Connectivity Establishment STUN + TURN candidates 85% direct P2P, 15% need relay
STUN Discover public IP/port behind NAT Query public STUN server Fails on symmetric NAT
TURN Media relay when direct fails Dedicated relay server Bandwidth-heavy, ~500 kbps per stream
SDP Media capability negotiation Codecs, formats, ICE candidates Offer/Answer model
SFU Selective Forwarding Unit Routes streams without decoding Bandwidth-efficient for group calls
MCU Multipoint Control Unit Mixes all streams into one High CPU, lower client bandwidth

Signaling Architecture: The Invisible Backbone

Signaling is the most underestimated part of WebRTC. The spec deliberately leaves it undefined, which means every team builds it differently — and most get it wrong.

WebSocket vs SSE vs HTTP Polling

WebSocket (best for bidirectional signaling):
┌──────────┐                         ┌──────────────┐
│ Client A │──ws://signaling:8080──▶│Signaling Server│◀──ws://signaling:8080──│ Client B │
└──────────┘                         └──────┬───────┘                         └──────────┘
                                            │
                                    ┌───────┴───────┐
                                    │   Room State   │
                                    │   Redis/Hazel   │
                                    └───────────────┘

Key signaling messages between two peers:

// Client A wants to call Client B
// 1. A → Server: create offer
socket.emit("call-offer", {
  to: "user-b",
  sdp: localDescription,  // Generated by RTCPeerConnection.createOffer()
});

// 2. Server → B: forward offer
socket.to("user-b").emit("call-offer", {
  from: "user-a",
  sdp: peerAOffer,
});

// 3. B → Server: send answer
socket.emit("call-answer", {
  to: "user-a",
  sdp: localDescription,  // Generated by RTCPeerConnection.createAnswer()
});

// 4. ICE trickling (happens in parallel)
// A → Server → B: ICE candidates as they're discovered
socket.emit("ice-candidate", {
  to: "user-b",
  candidate: iceCandidate,
});

Production Signaling Server (Node.js + Socket.IO)

// server/signaling.ts
import { Server } from "socket.io";
import { createAdapter } from "@socket.io/redis-adapter";
import { Redis } from "ioredis";

const io = new Server(3000, {
  pingInterval: 10000,
  pingTimeout: 5000,
  connectTimeout: 10000,
});

// Redis adapter for horizontal scaling
const pubClient = new Redis({ host: "redis" });
const subClient = pubClient.duplicate();
io.adapter(createAdapter(pubClient, subClient));

// Room management with Redis
interface RoomState {
  id: string;
  participants: Map<string, Participant>;
  maxParticipants: number;
}

io.on("connection", (socket) => {
  socket.on("join-room", async ({ roomId, userId }) => {
    // Check room capacity
    const count = await redis.scard(`room:${roomId}:participants`);
    if (count >= 12) {
      socket.emit("error", { message: "Room full" });
      return;
    }

    socket.join(roomId);
    await redis.sadd(`room:${roomId}:participants`, userId);

    // Notify others
    socket.to(roomId).emit("participant-joined", { userId });

    // Send existing participants to new joiner
    const participants = await redis.smembers(`room:${roomId}:participants`);
    socket.emit("room-state", { participants });
  });

  // Forward signaling messages
  socket.on("offer", ({ to, sdp }) => {
    io.to(to).emit("offer", { from: socket.id, sdp });
  });

  socket.on("answer", ({ to, sdp }) => {
    io.to(to).emit("answer", { from: socket.id, sdp });
  });

  socket.on("ice-candidate", ({ to, candidate }) => {
    io.to(to).emit("ice-candidate", { from: socket.id, candidate });
  });

  socket.on("disconnect", async () => {
    // Clean up room state
    const rooms = Array.from(socket.rooms);
    for (const roomId of rooms) {
      if (roomId !== socket.id) {
        await redis.srem(`room:${roomId}:participants`, socket.id);
        socket.to(roomId).emit("participant-left", { userId: socket.id });
      }
    }
  });
});

Signaling Gotchas

  1. Message ordering matters: If an ICE candidate arrives before the SDP answer, the browser ignores it. Implement a buffer: queue candidates, flush after remote description is set.
  2. Reconnection: Socket.IO handles transport reconnection, but you need application-level reconnection for signaling state. Store pending operations in Redis so a new server instance can resume.
  3. Stale state: If a participant disconnects without clean leave, their entry persists. Use TTL-expiring Redis keys for room membership.

NAT Traversal: The Ugly Reality

Why NAT Breaks P2P

Without NAT:
  Host A (1.2.3.4:50000) ←──────────→ Host B (5.6.7.8:60000)
  Direct UDP works fine.

With NAT:
  Host A (192.168.1.5:50000) ──NAT──▶ Public (1.2.3.4:12345)
  Host B (10.0.0.3:60000)    ──NAT──▶ Public (5.6.7.8:54321)
  
  A sends packet to 5.6.7.8:54321 → NAT B drops it (no mapping)

NATs only allow inbound packets from addresses the internal host has recently contacted. Two hosts behind NATs can't simply address each other — that's why ICE exists.

The ICE Process Step-by-Step

1. Gather Candidates
   └─ Host: 192.168.1.5:50000 (local, unlikely to work)
   └─ SRFLX (STUN): 1.2.3.4:12345 (server-reflexive, mapped public IP)
   └─ RELAY (TURN): turn.example.com:3478 (relay, always works)

2. Prioritize and Exchange
   └─ Local < SRFLX < Relay (priority order)
   └─ Exchange candidates via signaling channel

3. Connectivity Checks
   └─ Try each candidate pair (local↔local, srflx↔srflx, etc.)
   └─ Sends STUN binding requests to verify reachability
   └─ First working pair wins

4. Nomination
   └─ Select the highest-priority working pair
   └─ Media begins flowing

NAT Types and Their Impact

NAT Type P2P Possible? TURN Required? Prevalence
Full Cone Yes (with STUN) No Rare (home routers)
Restricted Cone Yes (with STUN) No Common (home routers)
Port Restricted Cone Yes (with STUN) No Very common
Symmetric NAT No Yes Corporate, mobile, CGNAT
CGNAT (Carrier-Grade) No Yes Mobile carriers, some ISPs

In practice, 10-15% of all peer connections require TURN relay. For mobile users on cellular networks, that number jumps to 25-30%. You cannot skip TURN in production.

TURN Server Configuration (coturn)

# /etc/coturn/turnserver.conf
listening-port=3478
tls-listening-port=5349
relay-ip=10.0.1.5
external-ip=203.0.113.5/10.0.1.5

# Authentication
lt-cred-mech
user=myapp:myapp_turn_secret
realm=myapp.com

# Performance
max-bps=2000000         # 2 Mbps per session
total-quota=0           # Unlimited
no-multicast-peers

# Security
no-loopback-peers
no-cli
stale-nonce=600

# TLS
cert=/etc/letsencrypt/live/turn.myapp.com/fullchain.pem
pkey=/etc/letsencrypt/live/turn.myapp.com/privkey.pem

# Monitoring
prometheus

TURN Sizing Formula

TURN bandwidth needed = 
  (peak concurrent users × TURN usage ratio × bitrate per stream × 2 streams)

Example: 10,000 concurrent users, 15% TURN ratio, 1 Mbps video
  = 10,000 × 0.15 × 1 × 2 = 3,000 Mbps = 3 Gbps

Server count = 3,000 Mbps / 1,000 Mbps per server = 3 TURN servers minimum

SFU Architecture: Beyond P2P

For any call with more than 3 participants, P2P mesh fails catastrophically:

P2P Mesh (4 participants):
  A ↔ B, A ↔ C, A ↔ D, B ↔ C, B ↔ D, C ↔ D = 6 connections
  Each participant sends to 3 peers = upload 3× bitrate
  
P2P Mesh (8 participants):
  28 connections total, each sends to 7 peers = 7× upload
  Mobile devices choke at 3-4 uploads

SFU (Selective Forwarding Unit):
  8 participants × 1 connection to SFU = 8 connections
  Each uploads once, SFU forwards to 7 others
  Upload: 1× bitrate. Download: 7× bitrate (but downstream bandwidth is abundant)

SFU Implementation with mediasoup

// server/sfu.ts
import * as mediasoup from "mediasoup";

const worker = await mediasoup.createWorker({
  logLevel: "warn",
  rtcMinPort: 40000,
  rtcMaxPort: 49999,
});

const router = await worker.createRouter({
  mediaCodecs: [
    {
      kind: "video",
      mimeType: "video/VP8",
      clockRate: 90000,
      parameters: { "x-google-start-bitrate": 1000 },
    },
    {
      kind: "video",
      mimeType: "video/H264",
      clockRate: 90000,
      parameters: {
        "packetization-mode": 1,
        "profile-level-id": "42e01f",
      },
    },
    { kind: "audio", mimeType: "audio/opus", clockRate: 48000, channels: 2 },
  ],
});

// When a participant joins
async function handleNewParticipant(transport, roomId) {
  // Create producer (uploads media to SFU)
  const producer = await transport.produce({
    kind: "video",
    rtpParameters: clientRtpParameters,
    // Enable simulcast: 3 quality layers
    encodings: [
      { rid: "h", maxBitrate: 1500000, scalabilityMode: "L1T3" }, // High
      { rid: "m", maxBitrate: 500000, scalabilityMode: "L1T3" },  // Medium
      { rid: "l", maxBitrate: 150000, scalabilityMode: "L1T3" },  // Low
    ],
  });

  // Subscribe new participant to existing producers
  for (const existingProducer of getRoomProducers(roomId)) {
    const consumer = await transport.consume({
      producerId: existingProducer.id,
      rtpCapabilities: clientRtpCapabilities,
      paused: false,
    });
  }

  // Subscribe existing participants to new producer
  broadcastNewProducer(roomId, producer);
}

SFU vs MCU Decision

Feature P2P Mesh SFU MCU
Server CPU Zero Low (forwarding only) High (decode + encode)
Server Bandwidth Zero High (N-1 × bitrate per stream) Low (1 × bitrate per user)
Client Upload (N-1) × bitrate 1 × bitrate 1 × bitrate
Client Download (N-1) × bitrate (N-1) × bitrate 1 × bitrate
Latency Lowest +5-20ms +50-200ms
Best for 2-3 participants 4-50 participants Large broadcasts

Rule of thumb: Use P2P for 1:1 calls, SFU for group calls (3-50 people), MCU for large broadcasts where client bandwidth is the constraint.

Media Optimization

Simulcast: One Stream, Three Qualities

The uploader sends three versions of the same video at different bitrates. The SFU selectively forwards each receiver the best quality their network can handle:

// Client side: send simulcast
const pc = new RTCPeerConnection();

const stream = await navigator.mediaDevices.getUserMedia({
  video: { width: 1280, height: 720 },
});

const transceiver = pc.addTransceiver(stream.getVideoTracks()[0], {
  sendEncodings: [
    { rid: "h", maxBitrate: 1500_000, scaleResolutionDownBy: 1 },   // 720p
    { rid: "m", maxBitrate: 500_000, scaleResolutionDownBy: 2 },    // 360p
    { rid: "l", maxBitrate: 150_000, scaleResolutionDownBy: 4 },    // 180p
  ],
});

Bandwidth Estimation

WebRTC uses Google Congestion Control (GCC), which combines:

  • Loss-based: If packet loss > 2-10%, reduce bitrate
  • Delay-based: If one-way delay increases, reduce bitrate before loss occurs

You can monitor this from getStats():

const stats = await pc.getStats();
stats.forEach((report) => {
  if (report.type === "remote-inbound-rtp") {
    console.log({
      packetsLost: report.packetsLost,
      jitter: report.jitter,
      roundTripTime: report.roundTripTime,
      fractionLost: report.fractionLost,
    });
  }
  if (report.type === "candidate-pair" && report.state === "succeeded") {
    console.log({
      availableOutgoingBitrate: report.availableOutgoingBitrate,
      currentRoundTripTime: report.currentRoundTripTime,
    });
  }
});

10 Common Production Pitfalls

Pitfall 1: No TURN Server

"I'll add TURN if we need it." You need it. 10-15% of users will fail to connect. On mobile, 25-30%. Budget for TURN bandwidth from day one.

Pitfall 2: Single TURN Server

One server = SPOF. Run at least 2 TURN servers in different regions. Use iceTransportPolicy: "all" (default) to let the browser choose.

Pitfall 3: Ignoring ICE Candidate Trickling

Don't wait for all candidates. Add each candidate as it arrives:

pc.onicecandidate = ({ candidate }) => {
  if (candidate) {
    signaling.send({ type: "ice-candidate", candidate });
  }
};

Pitfall 4: getUserMedia Without Constraints

// ❌ This can request a 4K camera on a phone
const stream = await navigator.mediaDevices.getUserMedia({ video: true });

// ✅ Specify reasonable constraints
const stream = await navigator.mediaDevices.getUserMedia({
  video: {
    width: { ideal: 1280 },
    height: { ideal: 720 },
    frameRate: { ideal: 30 },
    facingMode: "user",
  },
  audio: {
    echoCancellation: true,
    noiseSuppression: true,
  },
});

Pitfall 5: Not Handling getUserMedia Errors

Permission denied, device not found, device in use — these are normal. Every one needs a user-friendly message:

try {
  const stream = await navigator.mediaDevices.getUserMedia(constraints);
} catch (err) {
  switch (err.name) {
    case "NotAllowedError":
      showError("Camera/microphone access denied. Please enable in browser settings.");
      break;
    case "NotFoundError":
      showError("No camera or microphone found.");
      break;
    case "NotReadableError":
      showError("Camera/microphone is in use by another application.");
      break;
    default:
      showError("Unable to access media devices: " + err.message);
  }
}

Pitfall 6: Network Change During Call

WiFi → 4G mid-call. The ICE connection breaks. Solution: ICE restart:

pc.oniceconnectionstatechange = () => {
  if (pc.iceConnectionState === "disconnected") {
    // Temporary — wait for recovery
    setTimeout(() => {
      if (pc.iceConnectionState === "disconnected") {
        // Still disconnected — restart ICE
        pc.restartIce();
        // Create new offer with iceRestart: true
        const offer = await pc.createOffer({ iceRestart: true });
        await pc.setLocalDescription(offer);
        signaling.send({ type: "offer", sdp: pc.localDescription });
      }
    }, 5000);
  }
};

Pitfall 7: Browser-Specific Bugs

Browser Known Issue Workaround
Chrome Simulcast with VP9 sometimes drops layers Use VP8 or H264 for simulcast
Firefox No simulcast support for H264 Use VP8 for multi-party calls
Safari getUserMedia requires user gesture Wrap in click handler
Safari iOS Only one video element can play at a time Use canvas for thumbnails
Chrome Android Some devices cap at 720p Request 720p explicitly

Pitfall 8: Not Monitoring WebRTC Stats

Without stats, you have no idea what's happening. Export key metrics to your monitoring system:

setInterval(async () => {
  const stats = await pc.getStats();
  for (const [_, report] of stats) {
    if (report.type === "remote-inbound-rtp" && report.kind === "video") {
      metrics.gauge("webrtc.packet_loss", report.fractionLost * 100);
      metrics.gauge("webrtc.jitter", report.jitter * 1000);
      metrics.gauge("webrtc.bitrate", report.bitrate / 1000);
    }
  }
}, 2000); // Every 2 seconds

Pitfall 9: Excessive Bitrate

Default WebRTC can push 2.5 Mbps for 720p. On a poor connection, this causes congestion. Cap it:

const sender = pc.getSenders().find(s => s.track?.kind === "video");
const params = sender.getParameters();
params.encodings[0].maxBitrate = 1000_000; // Cap at 1 Mbps
await sender.setParameters(params);

Pitfall 10: Forgetting to Clean Up

Zombie PeerConnections leak memory. Always close:

function hangup() {
  pc.getSenders().forEach(s => pc.removeTrack(s));
  pc.getReceivers().forEach(r => pc.removeTrack(r));
  localStream.getTracks().forEach(t => t.stop());
  pc.close();
  pc = null;
}

Monitoring Dashboard Metrics

Metric What It Means Healthy Range Alert Threshold
ICE Connection Time Time to establish connection 1-3 seconds > 10 seconds
Packet Loss % packets lost 0-2% > 5%
Jitter Packet arrival variance 0-30ms > 50ms
RTT Round-trip time 20-80ms > 300ms
Frame Rate Video frames per second 25-30 < 15
Bitrate Current video bitrate 500-1500 kbps < 100 kbps (frozen)
TURN Usage % connections using relay 10-20% > 30% (CSTUN down?)

Summary: WebRTC production deployment is 20% media and 80% infrastructure. The three things that will cause the most outages: running without TURN (10-30% connection failures), not handling network changes (ICE restart is mandatory), and single server for everything (TURN, signaling, and SFU all need their own scaling strategy). Build monitoring from day one — WebRTC issues are invisible without stats. Budget for TURN bandwidth, test on real mobile networks, and never trust the demo that works on localhost.

Online Tools

Try these browser-local tools — no sign-up required →

#WebRTC#实时通信#STUN#TURN#SFU#音视频#2026