Multimodal LLM Deployment: Vision-Language Model Inference and Production

Summary

• Multimodal LLMs (VLMs) have moved from lab to production in 2026: Qwen2.5-VL, InternVL3, and LLaVA-OneVision form the dominant trio
• The VRAM bottleneck in VLM inference lies in the Vision Encoder: a single 1080p image can produce 576-2,048 visual tokens
• Dynamic resolution handling is the core challenge for VLMs: token counts can vary by 4x across different image sizes
• Visual token compression (Pooling/Projection) can reduce VLM inference latency by 40%+
• This article provides a complete solution from model selection to production deployment, including Qwen2.5-VL K8s deployment

---

Table of Contents

• Multimodal LLM Landscape 2026
• VLM Architecture: From Image to Token
• VLM Inference Optimization: Visual Token Compression
• Image Understanding Pipeline: From Upload to Answer
• Qwen2.5-VL Production Deployment
• Summary and Further Reading

---

Multimodal LLM Landscape 2026

Mainstream VLM Comparison

Dimension	Qwen2.5-VL-72B	InternVL3-78B	LLaVA-OneVision-72B	Gemini 2.5 Pro
Developer	Alibaba	Shanghai AI Lab	AI2	Google
LLM Backbone	Qwen2.5-72B	InternLM3	Qwen2-72B	Gemini
Vision Encoder	ViT (675M)	InternViT (6B)	SigLIP (0.4B)	Proprietary
Max Resolution	Dynamic (megapixels)	4K	768x768	Dynamic
Video Understanding	Yes	Yes	Yes	Yes
OCR	Yes Strong	Yes Strong	Average	Yes Strong
Open Source	Yes	Yes	Yes	No
MMBoard Score	82.5	83.1	79.8	85.2

VLM Selection Decision Guide

Scenario	Recommended Model	Reason
General Image Understanding	Qwen2.5-VL-7B	Easy deployment, strong Chinese
High-Accuracy OCR	InternVL3-26B	Best OCR capability
Video Understanding	Qwen2.5-VL-72B	Dynamic resolution + video
Edge Deployment	Qwen2.5-VL-3B	Small model, fast speed
Closed-Source API	Gemini 2.5 Pro	Best overall

---

VLM Architecture: From Image to Token

VLM 3-Stage Architecture

┌──────────────────────────────────────────────────────────────┐
│              VLM 3-Stage Architecture                         │
│                                                                │
│  Stage 1: Visual Encoding                                     │
│  ┌──────────┐   ┌──────────────┐   ┌──────────────────┐     │
│  │ Raw      │──→│ Vision       │──→│ Visual Token     │     │
│  │ Image    │   │ Encoder(ViT) │   │ Sequence         │     │
│  │ 1080p    │   │              │   │ 576-2048 tokens   │     │
│  └──────────┘   └──────────────┘   └──────────────────┘     │
│                         ↓                                      │
│  Stage 2: Vision-Language Projection                          │
│  ┌──────────────────┐   ┌──────────────────┐               │
│  │ Visual Token     │──→│ Projection       │──→ Language    │
│  │ Sequence         │   │ (MLP/Q-Former)   │   Space Token  │
│  │ 576-2048 tokens  │   │                  │               │
│  └──────────────────┘   └──────────────────┘               │
│                         ↓                                      │
│  Stage 3: Language Model Generation                           │
│  ┌──────────────────────────────────────────────────┐       │
│  │ LLM (Qwen2.5-7B)                                 │       │
│  │ Input: [Visual Tokens] + [Text Tokens]            │       │
│  │ Output: Text Response                             │       │
│  └──────────────────────────────────────────────────┘       │
└──────────────────────────────────────────────────────────────┘

Visual Token Count Calculation

Image Resolution	ViT Patch Size	Visual Token Count	VRAM Usage (FP16)
224x224	14x14	256	0.5MB
512x512	14x14	1,296	2.5MB
1080p	14x14	5,616	11MB
4K	14x14	22,528	44MB

---

VLM Inference Optimization: Visual Token Compression

3 Token Compression Strategies

Strategy	Compression Ratio	Accuracy Loss	Latency Reduction	Use Case
2x2 Pooling	4x	1-2%	35%	General recommendation
Projection Layer Compression	4-8x	2-3%	40%	High throughput scenarios
Dynamic Resolution	Adaptive	0%	20-50%	Mixed resolution

2x2 Pooling Implementation

import torch
import torch.nn as nn

class VisualTokenPooler(nn.Module):
    def __init__(self, pool_size: int = 2):
        super().__init__()
        self.pool_size = pool_size

    def forward(self, visual_tokens: torch.Tensor) -> torch.Tensor:
        batch_size, seq_len, hidden_dim = visual_tokens.shape
        h = w = int(seq_len ** 0.5)
        tokens = visual_tokens.view(batch_size, h, w, hidden_dim)
        tokens = tokens.permute(0, 3, 1, 2)
        pooled = nn.functional.avg_pool2d(tokens, kernel_size=self.pool_size)
        pooled = pooled.permute(0, 2, 3, 1)
        _, new_h, new_w, _ = pooled.shape
        return pooled.reshape(batch_size, new_h * new_w, hidden_dim)

Dynamic Resolution Handling

from qwen_vl_utils import process_vision_info

def prepare_vlm_inputs(image_path: str, question: str, tokenizer, max_pixels: int = 1003520):
    messages = [{
        "role": "user",
        "content": [
            {"type": "image", "image": image_path, "resized_height": None, "resized_width": None},
            {"type": "text", "text": question},
        ],
    }]

    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    image_inputs, video_inputs = process_vision_info(messages)

    inputs = tokenizer(
        text=[text],
        images=image_inputs,
        videos=video_inputs,
        padding=True,
        return_tensors="pt",
    ).to("cuda")

    return inputs

---

Image Understanding Pipeline: From Upload to Answer

Complete Pipeline Implementation

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import base64
import io
from PIL import Image

class ImageUnderstandingPipeline:
    def __init__(self, model_id: str = "Qwen/Qwen2.5-VL-7B-Instruct"):
        self.model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
            model_id,
            torch_dtype=torch.bfloat16,
            device_map="auto",
        )
        self.processor = AutoProcessor.from_pretrained(model_id)

    async def understand(self, image_base64: str, question: str) -> str:
        image_bytes = base64.b64decode(image_base64)
        image = Image.open(io.BytesIO(image_bytes))

        messages = [{
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": question},
            ],
        }]

        text = self.processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
        image_inputs, video_inputs = process_vision_info(messages)
        inputs = self.processor(
            text=[text],
            images=image_inputs,
            videos=video_inputs,
            padding=True,
            return_tensors="pt",
        ).to(self.model.device)

        output_ids = self.model.generate(**inputs, max_new_tokens=1024)
        generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(inputs.input_ids, output_ids)]
        output_text = self.processor.batch_decode(generated_ids, skip_special_tokens=True)

        return output_text[0]

    async def batch_understand(self, images: list[str], question: str) -> list[str]:
        tasks = [self.understand(img, question) for img in images]
        return await asyncio.gather(*tasks)

VLM Inference Performance Benchmarks

Model	GPU	Image Resolution	Prefill(s)	Decode(tok/s)	Total VRAM
Qwen2.5-VL-7B	A100x1	512x512	0.8	2800	18GB
Qwen2.5-VL-7B	A100x1	1080p	2.5	2200	24GB
Qwen2.5-VL-7B+Pool	A100x1	1080p	1.5	2600	20GB
Qwen2.5-VL-72B	H100x4	1080p	5.2	680	85GB

---

Qwen2.5-VL Production Deployment

vLLM VLM Deployment

python -m vllm.entrypoints.openai.api_server \
    --model Qwen/Qwen2.5-VL-7B-Instruct \
    --host 0.0.0.0 \
    --port 8000 \
    --tensor-parallel-size 2 \
    --gpu-memory-utilization 0.92 \
    --max-model-len 8192 \
    --limit-mm-per-prompt image=5 \
    --enable-prefix-caching

K8s Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: qwen25-vl-7b
  namespace: ai-inference
spec:
  replicas: 2
  selector:
    matchLabels:
      app: qwen25-vl-7b
  template:
    spec:
      containers:
        - name: vllm
          image: vllm/vllm-openai:v0.8.0
          ports:
            - containerPort: 8000
          resources:
            limits:
              nvidia.com/gpu: 2
            requests:
              nvidia.com/gpu: 2
              cpu: "4"
              memory: 16Gi
          args:
            - --model
            - Qwen/Qwen2.5-VL-7B-Instruct
            - --host
            - "0.0.0.0"
            - --port
            - "8000"
            - --tensor-parallel-size
            - "2"
            - --gpu-memory-utilization
            - "0.92"
            - --max-model-len
            - "8192"
            - --limit-mm-per-prompt
            - image=5
            - --enable-prefix-caching
          livenessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 180
            periodSeconds: 30

API Call Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")

with open("chart.png", "rb") as f:
    import base64
    image_b64 = base64.b64encode(f.read()).decode()

response = client.chat.completions.create(
    model="Qwen/Qwen2.5-VL-7B-Instruct",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_b64}"}},
            {"type": "text", "text": "Analyze the key trends in this chart"},
        ],
    }],
    max_tokens=1024,
)

print(response.choices[0].message.content)

---

Summary and Further Reading

Multimodal LLMs have moved from the lab to production. Qwen2.5-VL has become the go-to open-source VLM thanks to its dynamic resolution and strong OCR capabilities, while visual token compression can reduce inference latency by 40%+. The key to VLM deployment is balancing image resolution with token count.

Key Deployment Takeaways:

1. VLM Selection: Choose Qwen2.5-VL for general use, InternVL3 for OCR, and Gemini for closed-source
2. Visual tokens are the VRAM killer: a 1080p image produces 5,616 tokens
3. 2x2 Pooling is the most practical token compression strategy, with accuracy loss under 2%
4. vLLM natively supports VLM deployment; use --limit-mm-per-prompt to control concurrent image count
5. Dynamic resolution handling is the biggest difference between VLMs and text-only LLMs

Related Reading:

• LLM Inference Acceleration Benchmarks — VLM inference engine selection
• Python AI Multimodal RAG in Practice — VLM applications in multimodal RAG
• AI Chip Inference Deployment in Practice — VLM deployment on different chips

Authoritative References:

• Qwen2.5-VL Technical Report
• InternVL3