HTTP/3 & QUIC Debugging: 5 Production Patterns from Packet Capture to Performance Analysis
Why HTTP/3 Debugging Is 10x Harder Than HTTP/2
HTTP/3 runs on QUIC over UDP, which means 20 years of TCP debugging experience is essentially useless. TCP packets are directly readable in Wireshark, HTTP/2 streams are visible in Chrome DevTools — but QUIC traffic is encrypted, making packets unreadable; HTTP/3's stream multiplexing happens at the transport layer, invisible to application-layer tools.
In 2026, global HTTP/3 adoption exceeds 45% (Cloudflare Radar data), yet when developers encounter issues, they often see nothing more than "connection failed" with no way to investigate. This article summarizes 5 production-validated debugging patterns, from packet capture decryption to real-time monitoring, helping you build a complete HTTP/3 debugging toolkit.
| Debugging Dimension | HTTP/2 Tool | HTTP/3 Tool | Difficulty Change |
|---|---|---|---|
| Packet Capture | Wireshark (directly readable) | Wireshark + SSLKEYLOG | Requires key decryption |
| Protocol Logging | No standard | qlog (standardized format) | New concept |
| Client Debugging | Chrome DevTools | Chrome NetLog | Lower level |
| CLI Debugging | curl -v | curl --http3 + env vars | Requires compilation |
| Production Monitoring | TCP metrics | QUIC-specific metrics | Different metric system |
Pattern 1: Wireshark QUIC Packet Capture and Dissection
Core Problem: QUIC Is Fully Encrypted
QUIC integrates TLS 1.3 directly into the protocol. All frames, including headers, are encrypted. Wireshark captures of QUIC packets only show UDP payloads — internal HTTP/3 frames cannot be parsed. To decrypt QUIC traffic, you must obtain TLS session keys.
SSLKEYLOGFILE Mechanism
SSLKEYLOGFILE is the standard TLS debugging mechanism. Clients that support it (Chrome, Firefox, curl, Go) write TLS session keys to a specified file. Wireshark reads this file to decrypt traffic.
# Set SSLKEYLOGFILE environment variable
export SSLKEYLOGFILE=/tmp/sslkeys.log
# Capture HTTP/3 traffic with curl (keys written automatically)
curl --http3 https://example.com -v
# Access with Chrome (keys written automatically)
google-chrome --ssl-key-log-file=/tmp/sslkeys.log
# Access with Firefox
export MOZ_LOG="ssl:5"
export SSLKEYLOGFILE=/tmp/sslkeys.log
firefox
Wireshark QUIC Decryption Configuration
# 1. Start Wireshark capture (filter QUIC traffic)
# Capture filter:
udp port 443
# 2. Configure key log file in Wireshark
# Edit -> Preferences -> Protocols -> TLS -> (Pre)-Master-Secret log filename
# Enter: /tmp/sslkeys.log
# 3. Verify QUIC decryption success
# Before decryption: QUIC packets show "Protected Payload, PKN: ..."
# After decryption: HTTP/3 frames visible (HEADERS, DATA, SETTINGS, etc.)
tshark Command-Line Packet Analysis
# Capture QUIC traffic with decryption
tshark -i eth0 -f "udp port 443" \
-o "tls.keylog_file:/tmp/sslkeys.log" \
-Y "quic" \
-T fields \
-e quic.packet_type \
-e quic.frame_type \
-e ip.src \
-e ip.dst \
-e quic.stream_id
# Extract HTTP/3 request headers
tshark -i eth0 -f "udp port 443" \
-o "tls.keylog_file:/tmp/sslkeys.log" \
-Y "http3" \
-T fields \
-e http3.header \
-e http3.stream_id
# Analyze QUIC connection handshake information
tshark -i eth0 -f "udp port 443" \
-o "tls.keylog_file:/tmp/sslkeys.log" \
-Y "quic.connection_id" \
-T fields \
-e quic.connection_id \
-e quic.version
# Analyze QUIC packet loss and retransmission
tshark -i eth0 -f "udp port 443" \
-o "tls.keylog_file:/tmp/sslkeys.log" \
-Y "quic.frame_type == 0x02 || quic.frame_type == 0x03" \
-T fields \
-e quic.frame_type \
-e quic.ack_range_count \
-e quic.ack_delay
Go Server Key Export
package main
import (
"crypto/tls"
"fmt"
"log"
"net/http"
"os"
)
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello HTTP/3!")
})
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{loadCert()},
NextProtos: []string{"h3"},
KeyLogWriter: keyLogWriter(),
}
server := &http.Server{
Addr: ":443",
Handler: mux,
TLSConfig: tlsConfig,
}
log.Fatal(server.ListenAndServeTLS("", ""))
}
func keyLogWriter() tls.KeyLogWriter {
f, err := os.OpenFile("/tmp/server-sslkeys.log", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0600)
if err != nil {
log.Printf("Failed to open keylog file: %v", err)
return nil
}
return f
}
func loadCert() tls.Certificate {
cert, err := tls.LoadX509KeyPair("server.crt", "server.key")
if err != nil {
log.Fatal(err)
}
return cert
}
QUIC Version Identification
# Common QUIC version numbers
# RFC 9000 (QUIC v1): 0x00000001
# RFC 9369 (QUIC v2): 0x6b3343cf
# Google QUIC (gQUIC): 0x51303039 (Q039)
# IETF Draft 29: 0xff00001d
# Filter specific version with tshark
tshark -i eth0 -f "udp port 443" \
-Y "quic.version == 0x00000001" \
-T fields \
-e quic.connection_id \
-e quic.version
Pattern 2: qlog Analysis and Visualization
What Is qlog
qlog is the IETF-standardized QUIC/HTTP/3 logging format (RFC 9657). It defines a unified event model that enables interoperability across different QUIC implementations. Unlike proprietary log formats, qlog lets you analyze logs from quiche, lsquic, quic-go, ngtcp2, and other implementations with the same tools.
{
"qlog_version": "0.3",
"title": "QUIC Connection Debug Log",
"description": "HTTP/3 connection from client to example.com",
"trace": [
{
"vantage_point": {
"type": "client"
},
"title": "Client trace",
"event_fields": [
"relative_time",
"category",
"event_type",
"data"
],
"events": [
[0, "transport", "packet_sent", {
"packet_type": "initial",
"header": {
"packet_number": 0,
"version": "0x00000001",
"scid": "0x8394c8f03e515708",
"dcid": "0x06b8a0b3a1914fc2"
},
"frames": [
{
"frame_type": "crypto",
"offset": 0,
"length": 387
},
{
"frame_type": "padding"
}
]
}],
[0.536, "transport", "packet_received", {
"packet_type": "initial",
"header": {
"packet_number": 0
},
"frames": [
{
"frame_type": "crypto",
"offset": 0,
"length": 1256
}
]
}],
[1.024, "http", "frame_created", {
"stream_id": 0,
"frame": {
"frame_type": "headers",
"headers": [
{"name": ":method", "value": "GET"},
{"name": ":path", "value": "/"},
{"name": ":authority", "value": "example.com"},
{"name": ":scheme", "value": "https"}
]
}
}]
]
}
]
}
Server-Side qlog Collection
# Nginx QUIC qlog configuration (1.27+)
http {
quic_log_dir /var/log/nginx/quic;
quic_log_level debug;
server {
listen 443 quic reuseport;
server_name example.com;
access_log /var/log/nginx/quic/access.log quic_format;
}
}
quic-go qlog Collection
package main
import (
"context"
"encoding/json"
"log"
"net"
"os"
"time"
"github.com/quic-go/quic-go"
"github.com/quic-go/quic-go/logging"
)
type qlogTracer struct {
file *os.File
}
func newQlogTracer(filepath string) (*qlogTracer, error) {
f, err := os.Create(filepath)
if err != nil {
return nil, err
}
return &qlogTracer{file: f}, nil
}
func (t *qlogTracer) Trace() *logging.ConnectionTracer {
return &logging.ConnectionTracer{
StartedConnection: func(local, remote net.Addr, srcConnID, destConnID logging.ConnectionID) {
event := map[string]interface{}{
"time": time.Now().Format(time.RFC3339Nano),
"category": "transport",
"event": "connection_started",
"data": map[string]interface{}{
"src_cid": srcConnID.String(),
"dest_cid": destConnID.String(),
},
}
t.writeEvent(event)
},
ReceivedPacket: func(hdr *logging.PacketHeader, size logging.ByteCount, frames []logging.Frame) {
event := map[string]interface{}{
"time": time.Now().Format(time.RFC3339Nano),
"category": "transport",
"event": "packet_received",
"data": map[string]interface{}{
"packet_type": hdr.Type.String(),
"packet_size": size,
"frame_count": len(frames),
},
}
t.writeEvent(event)
},
SentPacket: func(hdr *logging.PacketHeader, size logging.ByteCount, frames []logging.Frame) {
event := map[string]interface{}{
"time": time.Now().Format(time.RFC3339Nano),
"category": "transport",
"event": "packet_sent",
"data": map[string]interface{}{
"packet_type": hdr.Type.String(),
"packet_size": size,
"frame_count": len(frames),
},
}
t.writeEvent(event)
},
}
}
func (t *qlogTracer) writeEvent(event map[string]interface{}) {
data, _ := json.Marshal(event)
t.file.Write(data)
t.file.Write([]byte("\n"))
}
func main() {
tracer, err := newQlogTracer("/tmp/quic-connection.qlog")
if err != nil {
log.Fatal(err)
}
defer tracer.file.Close()
tlsConfig := &tls.Config{
NextProtos: []string{"h3"},
}
quicConfig := &quic.Config{
Tracer: func(ctx context.Context, p logging.Perspective, ci logging.ConnectionID) logging.ConnectionTracer {
return *tracer.Trace()
},
}
conn, err := quic.DialAddr(context.Background(), "example.com:443", tlsConfig, quicConfig)
if err != nil {
log.Fatal(err)
}
defer conn.Close()
log.Printf("Connected via QUIC, version: %s", conn.ConnectionState().Version)
}
qlog Visualization Tools
# qlog visualization: qvis
# Online tool: https://qvis.quictools.info/
# Upload .qlog file for visualization
# Install qlog-converter
npm install -g qlog-converter
# Convert binary qlog to JSON format
qlog-converter -i binary.qlog -o json.qlog --format JSON
# Convert JSON qlog to readable text
qlog-converter -i json.qlog -o readable.txt --format TEXT
# Generate visualization report
npm install -g qlog-visualizer
qlog-visualizer -i connection.qlog -o report.html
qlog Key Event Analysis
# Analyze handshake timing
cat connection.qlog | python3 -c "
import json, sys
data = json.load(sys.stdin)
events = data['trace'][0]['events']
handshake_events = [e for e in events if e[1] == 'transport' and 'packet' in e[2]]
for e in handshake_events[:10]:
print(f't={e[0]:.3f}s {e[2]} type={e[3].get(\"packet_type\", \"?\")}')
"
# Analyze packet loss and retransmission
cat connection.qlog | python3 -c "
import json, sys
data = json.load(sys.stdin)
events = data['trace'][0]['events']
loss_events = [e for e in events if 'loss' in str(e[2]).lower() or 'retransmit' in str(e[2]).lower()]
for e in loss_events:
print(f't={e[0]:.3f}s {e[2]} data={e[3]}')
print(f'Total loss events: {len(loss_events)}')
"
# Analyze stream-level timing
cat connection.qlog | python3 -c "
import json, sys
data = json.load(sys.stdin)
events = data['trace'][0]['events']
stream_events = [e for e in events if e[1] == 'http' and 'stream' in str(e[3])]
for e in stream_events:
stream_id = e[3].get('stream_id', '?')
print(f't={e[0]:.3f}s stream={stream_id} {e[2]}')
"
Pattern 3: Chrome NetLog for HTTP/3 Debugging
Why DevTools Isn't Enough
Chrome DevTools' Network panel only shows application-layer information (request headers, response bodies, timing). It cannot see QUIC transport-layer details (connection migration, 0-RTT status, stream priorities, loss recovery). To debug HTTP/3 transport issues, you must use Chrome NetLog.
Launching Chrome with NetLog Capture
# Method 1: Command-line flags
google-chrome \
--enable-logging=netlog \
--net-log-capture-mode=Everything \
--net-log=/tmp/chrome-netlog.json
# Method 2: chrome://net-internals real-time view
# 1. Navigate to chrome://net-internals/#export
# 2. Click "Start logging to disk"
# 3. Perform the operation you want to debug
# 4. Click "Stop logging"
# Method 3: Chrome DevTools Protocol
google-chrome --remote-debugging-port=9222
# Trigger NetLog via CDP
curl -s http://localhost:9222/json/version | python3 -m json.tool
NetLog Event Analysis
# NetLog output is JSON format containing all network events
# Typical structure:
# {
# "constants": { ... },
# "events": [
# {"time": ..., "type": "HTTP3_SESSION_INITIALIZED", ...},
# {"time": ..., "type": "QUIC_SESSION_PACKET_SENT", ...},
# ...
# ]
# }
# Extract HTTP/3 related events
cat /tmp/chrome-netlog.json | python3 -c "
import json, sys
data = json.load(sys.stdin)
events = data.get('events', [])
http3_events = [e for e in events if 'HTTP3' in e.get('type', '') or 'QUIC' in e.get('type', '')]
for e in http3_events[:50]:
print(f't={e.get(\"time\",0)/1000000:.3f}s type={e.get(\"type\",\"?\")}')
"
# Analyze QUIC connection establishment timing
cat /tmp/chrome-netlog.json | python3 -c "
import json, sys
data = json.load(sys.stdin)
events = data.get('events', [])
connect_events = [e for e in events if any(k in e.get('type','') for k in ['QUIC_CONNECT', 'QUIC_SESSION_CONNECT', 'HTTP3_SESSION'])]
for e in connect_events:
t = e.get('time', 0) / 1000000
print(f't={t:.3f}s {e.get(\"type\",\"?\")}')
params = e.get('params', {})
if params:
for k, v in params.items():
if k in ['host', 'quic_version', 'connection_id', 'error', 'is_alternative_service']:
print(f' {k}={v}')
"
# Check 0-RTT status
cat /tmp/chrome-netlog.json | python3 -c "
import json, sys
data = json.load(sys.stdin)
events = data.get('events', [])
zrtt_events = [e for e in events if 'ZERO_RTT' in e.get('type', '') or 'early_data' in str(e.get('params', {})).lower()]
for e in zrtt_events:
print(f't={e.get(\"time\",0)/1000000:.3f}s {e.get(\"type\")} params={e.get(\"params\",{})}')
if not zrtt_events:
print('No 0-RTT events found')
"
# Analyze connection migration
cat /tmp/chrome-netlog.json | python3 -c "
import json, sys
data = json.load(sys.stdin)
events = data.get('events', [])
migration_events = [e for e in events if 'MIGRATION' in e.get('type', '') or 'CONNECTION_MIGRATION' in e.get('type', '')]
for e in migration_events:
print(f't={e.get(\"time\",0)/1000000:.3f}s {e.get(\"type\")} params={e.get(\"params\",{})}')
if not migration_events:
print('No connection migration events found')
"
chrome://net-internals Real-Time Debugging
chrome://net-internals key pages:
#h3 - HTTP/3 session list
#quic - QUIC connection list and configuration
#sockets - UDP socket state
#dns - DNS resolution (including HTTPS records)
#httpCache - Cache state
#altSvc - Alt-Svc cache contents
# Through chrome://net-internals/#quic view:
# - Active QUIC connections
# - Version, CID, status for each connection
# - QUIC configuration parameters
# - Connection error information
# Through chrome://net-internals/#h3 view:
# - HTTP/3 session state
# - Stream creation and closure
# - Priority dependency relationships
# - Push state
Alt-Svc Cache Debugging
# View Alt-Svc cache
# chrome://net-internals/#altSvc
# Common issue: Alt-Svc cache expired or incorrect
# Clear Alt-Svc cache:
# 1. Open chrome://net-internals/#altSvc
# 2. Click "Clear alt-svc cache"
# 3. Revisit the target website
# Force HTTP/3 (skip Alt-Svc discovery)
google-chrome \
--origin-to-force-quic-on=example.com:443 \
--net-log=/tmp/chrome-forced-h3.json
Pattern 4: curl HTTP/3 Debugging
Compiling curl with HTTP/3 Support
# Ubuntu 22.04+ compile curl with HTTP/3
# Method: boringssl + nghttp3 + ngtcp2
# Install dependencies
sudo apt-get install -y build-essential cmake git
# Compile boringssl
git clone https://boringssl.googlesource.com/boringssl
cd boringssl
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_POSITION_INDEPENDENT_CODE=on .
make -j$(nproc)
cd ..
# Compile ngtcp2
git clone https://github.com/ngtcp2/ngtcp2
cd ngtcp2
autoreconf -fi
./configure --with-boringssl=$(pwd)/../boringssl \
BORINGSSL_CFLAGS="-I$(pwd)/../boringssl/include" \
BORINGSSL_LIBS="-L$(pwd)/../boringssl/build/ssl -L$(pwd)/../boringssl/build/crypto -lssl -lcrypto"
make -j$(nproc)
sudo make install
cd ..
# Compile nghttp3
git clone https://github.com/ngtcp2/nghttp3
cd nghttp3
autoreconf -fi
./configure
make -j$(nproc)
sudo make install
cd ..
# Compile curl with HTTP/3
git clone https://github.com/curl/curl
cd curl
autoreconf -fi
./configure --with-openssl=$(pwd)/../boringssl \
--with-ngtcp2=$(pwd)/../ngtcp2 \
--with-nghttp3=$(pwd)/../nghttp3 \
LDFLAGS="-Wl,-rpath,$(pwd)/../boringssl/build/ssl:$(pwd)/../boringssl/build/crypto"
make -j$(nproc)
sudo make install
cd ..
# Verify
curl --version | grep -i http3
# Output should include: Features: ... HTTP3 ...
Basic HTTP/3 Request Debugging
# Send HTTP/3 request with verbose output
curl --http3 https://example.com -v
# Output example:
# * Trying 93.184.216.34:443...
# * Connected to example.com (93.184.216.34) port 443
# * QUIC handshake successful
# * Connection #0 to host example.com left intact
# > GET / HTTP/3
# > Host: example.com
# > user-agent: curl/8.7.1
# > accept: */*
# >
# < HTTP/3 200
# < content-type: text/html; charset=UTF-8
# < date: Mon, 16 Jun 2026 10:00:00 GMT
# Force HTTP/3 only (no fallback)
curl --http3-only https://example.com -v
# Test 0-RTT
# First request: normal handshake
curl --http3 https://example.com -w "time_connect: %{time_connect}\ntime_appconnect: %{time_appconnect}\ntime_total: %{time_total}\n" -o /dev/null -s
# Second request: 0-RTT (reuse session)
curl --http3 https://example.com -w "time_connect: %{time_connect}\ntime_appconnect: %{time_appconnect}\ntime_total: %{time_total}\n" -o /dev/null -s
curl Timing Analysis
# Complete timing output
curl --http3 https://example.com \
-w "\n=== Timing Breakdown ===\n\
namelookup: %{time_namelookup}s\n\
connect: %{time_connect}s\n\
appconnect: %{time_appconnect}s\n\
pretransfer: %{time_pretransfer}s\n\
starttransfer: %{time_starttransfer}s\n\
total: %{time_total}s\n\
\n=== Connection Info ===\n\
remote_ip: %{remote_ip}\n\
remote_port: %{remote_port}\n\
scheme: %{scheme}\n\
http_version: %{http_version}\n\
" -o /dev/null -s
# Compare HTTP/2 vs HTTP/3 connection time
echo "=== HTTP/2 ==="
curl --http2 https://example.com \
-w "appconnect: %{time_appconnect}s starttransfer: %{time_starttransfer}s total: %{time_total}s\n" \
-o /dev/null -s
echo "=== HTTP/3 ==="
curl --http3 https://example.com \
-w "appconnect: %{time_appconnect}s starttransfer: %{time_starttransfer}s total: %{time_total}s\n" \
-o /dev/null -s
# Batch test (10 runs, average)
for proto in h2 h3; do
total=0
for i in $(seq 1 10); do
t=$(curl --http${proto} https://example.com -w "%{time_total}" -o /dev/null -s 2>/dev/null)
total=$(echo "$total + $t" | bc)
done
avg=$(echo "scale=3; $total / 10" | bc)
echo "HTTP/${proto} average total time: ${avg}s"
done
curl Environment Variable Debugging
# Enable QUIC internal logging
export SSLKEYLOGFILE=/tmp/curl-quic-keys.log
# Enable ngtcp2 verbose logging
export NGTCP2_DEBUG_LOG=1
# Enable nghttp3 verbose logging
export NGHTTP3_DEBUG_LOG=1
# Send request
curl --http3 https://example.com -v 2>&1 | tee /tmp/curl-h3-debug.log
# Analyze log
grep -i "handshake\|0-rtt\|stream\|frame" /tmp/curl-h3-debug.log
# Test specific QUIC version
curl --http3 https://example.com -v \
--quic-version v1 # RFC 9000 (QUIC v1)
curl --http3 https://example.com -v \
--quic-version v2 # RFC 9369 (QUIC v2)
curl Simulating Network Issues
# Simulate high latency
curl --http3 https://example.com -v \
--limit-rate 100k \
--connect-timeout 5 \
--max-time 30
# Test connection timeout handling
curl --http3-only https://unreachable.example.com -v \
--connect-timeout 3 \
--max-time 10
# Test Alt-Svc discovery
# First get Alt-Svc header via HTTP/2
curl --http2 https://example.com -v -I 2>&1 | grep -i alt-svc
# Then manually connect via HTTP/3
curl --http3 https://example.com -v
# Send large file to test flow control
dd if=/dev/urandom bs=1M count=100 2>/dev/null | \
curl --http3 https://example.com/upload -v \
-X POST \
-H "Content-Type: application/octet-stream" \
--data-binary @-
Pattern 5: Production QUIC Monitoring
Prometheus QUIC Metrics Collection
package main
import (
"log"
"net/http"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
var (
quicConnectionsTotal = prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "quic_connections_total",
Help: "Total number of QUIC connections",
},
[]string{"version", "status"},
)
quicConnectionDuration = prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "quic_connection_duration_seconds",
Help: "QUIC connection duration in seconds",
Buckets: prometheus.ExponentialBuckets(0.1, 2, 15),
},
[]string{"version"},
)
quicHandshakeDuration = prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "quic_handshake_duration_seconds",
Help: "QUIC handshake duration in seconds",
Buckets: prometheus.ExponentialBuckets(0.001, 2, 15),
},
[]string{"version", "zero_rtt"},
)
quicStreamsTotal = prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "quic_streams_total",
Help: "Total number of QUIC streams",
},
[]string{"direction", "stream_type"},
)
quicPacketsLost = prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "quic_packets_lost_total",
Help: "Total number of QUIC packets lost",
},
[]string{"packet_type"},
)
quicRetransmitPackets = prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "quic_retransmit_packets_total",
Help: "Total number of QUIC retransmit packets",
},
[]string{"packet_type"},
)
quicBytesTransferred = prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "quic_bytes_transferred_total",
Help: "Total bytes transferred over QUIC",
},
[]string{"direction"},
)
quicConnectionMigrations = prometheus.NewCounter(
prometheus.CounterOpts{
Name: "quic_connection_migrations_total",
Help: "Total number of QUIC connection migrations",
},
)
quicZeroRTTAccepts = prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "quic_zero_rtt_accepts_total",
Help: "Total number of 0-RTT connection attempts",
},
[]string{"status"},
)
quicCongestionWindow = prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "quic_congestion_window_bytes",
Help: "Current QUIC congestion window size in bytes",
},
[]string{"connection_id"},
)
quicRtt = prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "quic_rtt_seconds",
Help: "QUIC round trip time in seconds",
Buckets: prometheus.ExponentialBuckets(0.001, 2, 15),
},
[]string{"rtt_type"},
)
)
func init() {
prometheus.MustRegister(
quicConnectionsTotal,
quicConnectionDuration,
quicHandshakeDuration,
quicStreamsTotal,
quicPacketsLost,
quicRetransmitPackets,
quicBytesTransferred,
quicConnectionMigrations,
quicZeroRTTAccepts,
quicCongestionWindow,
quicRtt,
)
}
func main() {
http.Handle("/metrics", promhttp.Handler())
mux := http.NewServeMux()
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.ProtoMajor == 3 {
quicConnectionsTotal.WithLabelValues("v1", "active").Inc()
}
w.WriteHeader(http.StatusOK)
w.Write([]byte("Hello HTTP/3!"))
})
log.Println("Starting server on :443 with metrics on :9090/metrics")
go func() {
log.Fatal(http.ListenAndServe(":9090", nil))
}()
log.Fatal(http.ListenAndServeTLS(":443", "server.crt", "server.key", mux))
}
Nginx QUIC Metrics Export
# nginx.conf - QUIC metrics via stub_status
http {
server {
listen 443 quic reuseport;
listen 443 ssl;
server_name example.com;
location /nginx_status {
stub_status;
allow 127.0.0.1;
deny all;
}
log_format quic_metrics
'$remote_addr '
'$quic_connection_id '
'$quic_version '
'$request_time '
'$upstream_response_time '
'$bytes_sent '
'$bytes_received '
'$status';
access_log /var/log/nginx/quic-metrics.log quic_metrics;
}
}
# Use nginx-prometheus-exporter to collect Nginx metrics
docker run -d --name nginx-exporter \
-p 9113:9113 \
nginx/nginx-prometheus-exporter:1.3 \
--nginx.scrape-uri=http://nginx:80/nginx_status
# Custom QUIC log parser (Python)
cat << 'EOF' > /usr/local/bin/quic-log-parser.py
import sys
from prometheus_client import Counter, Gauge, start_http_server
quic_connections = Counter('nginx_quic_connections', 'Nginx QUIC connections')
quic_request_duration = Gauge('nginx_quic_request_duration_seconds', 'QUIC request duration')
for line in sys.stdin:
parts = line.strip().split()
if len(parts) >= 8:
quic_connections.inc()
try:
duration = float(parts[3])
quic_request_duration.set(duration)
except ValueError:
pass
start_http_server(9114)
EOF
Grafana Dashboard Configuration
{
"dashboard": {
"title": "HTTP/3 & QUIC Production Monitoring",
"panels": [
{
"title": "QUIC Connection Rate",
"type": "timeseries",
"targets": [
{
"expr": "rate(quic_connections_total[5m])",
"legendFormat": "{{version}} {{status}}"
}
]
},
{
"title": "Handshake Duration",
"type": "timeseries",
"targets": [
{
"expr": "histogram_quantile(0.50, rate(quic_handshake_duration_seconds_bucket[5m]))",
"legendFormat": "p50 {{version}} 0-rtt={{zero_rtt}}"
},
{
"expr": "histogram_quantile(0.99, rate(quic_handshake_duration_seconds_bucket[5m]))",
"legendFormat": "p99 {{version}} 0-rtt={{zero_rtt}}"
}
]
},
{
"title": "Packet Loss Rate",
"type": "timeseries",
"targets": [
{
"expr": "rate(quic_packets_lost_total[5m]) / rate(quic_connections_total[5m])",
"legendFormat": "{{packet_type}} loss rate"
}
]
},
{
"title": "0-RTT Success Rate",
"type": "gauge",
"targets": [
{
"expr": "rate(quic_zero_rtt_accepts_total{status=\"accepted\"}[5m]) / rate(quic_zero_rtt_accepts_total[5m]) * 100",
"legendFormat": "0-RTT Accept %"
}
]
},
{
"title": "Connection Migrations",
"type": "stat",
"targets": [
{
"expr": "rate(quic_connection_migrations_total[1h])",
"legendFormat": "migrations/hour"
}
]
},
{
"title": "QUIC RTT Distribution",
"type": "timeseries",
"targets": [
{
"expr": "histogram_quantile(0.50, rate(quic_rtt_seconds_bucket[5m]))",
"legendFormat": "p50 {{rtt_type}}"
},
{
"expr": "histogram_quantile(0.95, rate(quic_rtt_seconds_bucket[5m]))",
"legendFormat": "p95 {{rtt_type}}"
}
]
},
{
"title": "Throughput",
"type": "timeseries",
"targets": [
{
"expr": "rate(quic_bytes_transferred_total{direction=\"send\"}[5m]) * 8 / 1000000",
"legendFormat": "Upload Mbps"
},
{
"expr": "rate(quic_bytes_transferred_total{direction=\"receive\"}[5m]) * 8 / 1000000",
"legendFormat": "Download Mbps"
}
]
},
{
"title": "Retransmit Rate",
"type": "timeseries",
"targets": [
{
"expr": "rate(quic_retransmit_packets_total[5m])",
"legendFormat": "{{packet_type}} retransmits/s"
}
]
}
]
}
}
Alerting Rules
# Prometheus alerting rules for QUIC
groups:
- name: quic_alerts
rules:
- alert: QUICHighPacketLoss
expr: rate(quic_packets_lost_total[5m]) / rate(quic_connections_total[5m]) > 0.05
for: 10m
labels:
severity: warning
annotations:
summary: "QUIC packet loss rate exceeds 5%"
description: "QUIC packet loss rate is {{ $value | humanizePercentage }} on {{ $labels.packet_type }}"
- alert: QUICHandshakeTimeout
expr: histogram_quantile(0.99, rate(quic_handshake_duration_seconds_bucket[5m])) > 1
for: 5m
labels:
severity: critical
annotations:
summary: "QUIC handshake p99 latency exceeds 1s"
description: "QUIC {{ $labels.version }} handshake p99 is {{ $value }}s"
- alert: QUICZeroRTTRejectionHigh
expr: rate(quic_zero_rtt_accepts_total{status="rejected"}[5m]) / rate(quic_zero_rtt_accepts_total[5m]) > 0.3
for: 15m
labels:
severity: warning
annotations:
summary: "QUIC 0-RTT rejection rate exceeds 30%"
description: "0-RTT rejection rate is {{ $value | humanizePercentage }}"
- alert: QUICConnectionFailureSpike
expr: rate(quic_connections_total{status="failed"}[5m]) > rate(quic_connections_total{status="active"}[5m]) * 0.1
for: 5m
labels:
severity: critical
annotations:
summary: "QUIC connection failure rate exceeds 10%"
description: "QUIC connection failures are {{ $value | humanizePercentage }} of successful connections"
- alert: QUICHighRetransmitRate
expr: rate(quic_retransmit_packets_total[5m]) / rate(quic_bytes_transferred_total[5m]) > 0.1
for: 10m
labels:
severity: warning
annotations:
summary: "QUIC retransmit rate exceeds 10%"
description: "QUIC retransmit rate is {{ $value | humanizePercentage }}"
5 Debugging Patterns Comparison
| Pattern | Use Case | Strengths | Limitations | Learning Curve |
|---|---|---|---|---|
| Wireshark Capture | Protocol-level issues, handshake analysis | Deepest, most complete | Requires keys, no live analysis | High |
| qlog Analysis | Cross-implementation comparison, optimization | Standardized, visualizable | Requires implementation support | Medium |
| Chrome NetLog | Client issues, connection migration | Browser-native, real-time | Chrome only, large data volume | Medium |
| curl Debugging | Quick validation, CI integration | CLI, scriptable | Requires compilation, limited features | Low |
| Production Monitoring | Production alerting, trend analysis | Real-time, global view | Requires instrumentation, no detail recall | Medium-High |
10 Common Debugging Scenarios Quick Reference
| # | Scenario | Recommended Pattern | Key Command/Tool |
|---|---|---|---|
| 1 | QUIC handshake failure | Wireshark | tshark -f "udp port 443" -o "tls.keylog_file:keys.log" -Y "quic.packet_type==initial" |
| 2 | 0-RTT rejected | Chrome NetLog | chrome://net-internals/#quic find ZERO_RTT events |
| 3 | Connection migration not working | Chrome NetLog | --origin-to-force-quic-on + NetLog MIGRATION events |
| 4 | Slow loss recovery | qlog | qvis visualize loss detection and recovery events |
| 5 | Stream priority issues | Wireshark | Decrypt and analyze HTTP/3 PRIORITY frames |
| 6 | Alt-Svc not working | curl | curl -I https://example.com check Alt-Svc header |
| 7 | Performance comparison | curl | curl --http2 vs curl --http3 timing comparison |
| 8 | Abnormal production packet loss | Prometheus | rate(quic_packets_lost_total[5m]) alert |
| 9 | Elevated handshake latency | Prometheus | histogram_quantile(0.99, quic_handshake_duration_seconds) |
| 10 | 0-RTT replay risk | Wireshark + qlog | Capture and verify 0-RTT data for non-idempotent requests |
Recommended Tools
During HTTP/3 debugging, these tools can help you analyze and verify:
- HTTP Status Code Lookup: Use /en/network/http-status to look up QUIC/HTTP3-related status code meanings
- Base64 Encoder: Use /en/encode/base64 for encoding/decoding certificates and tokens
- Hash Calculator: Use /en/encode/hash to compute hash values for QUIC configuration integrity
Summary: The core challenge of HTTP/3 debugging lies in QUIC's full encryption and UDP transport. The 5 production debugging patterns each have different focuses: Wireshark capture for protocol-level deep analysis (requires SSLKEYLOGFILE decryption), qlog for standardized cross-implementation log analysis, Chrome NetLog for client-side real-time debugging, curl for quick validation and CI integration, and Prometheus+Grafana for production continuous monitoring. Start with curl for quick validation, escalate to Wireshark/qlog for deep issues, and always build a complete QUIC metrics monitoring and alerting system in production.
Try these browser-local tools — no sign-up required →