BFF與AI Gateway架構：2026年統一LLM接入層設計

AI時代，為什麼需要AI Gateway？

當你的系統同時接入OpenAI、Claude、Gemini、通義千問、DeepSeek……每個LLM有不同的API格式、計費方式、限流策略。沒有統一接入層，你的業務程式碼會被LLM供應商徹底綁架。

真實案例：某公司從OpenAI遷移到Claude，因為API格式不同，改動了47個檔案，耗時2週。有了AI Gateway，遷移只需改1行配置。

BFF模式的三次進化

傳統BFF（2018）
  為不同前端聚合後端API，解決過度獲取問題

AI-Enhanced BFF（2024）
  BFF層加入AI能力：摘要、翻譯、內容生成
  AI只是BFF的一個下游服務

AI Gateway（2026）
  AI成為核心，BFF圍繞AI重構
  統一接入多LLM，管理路由、計費、安全
  業務程式碼只對接AI Gateway，不直接呼叫LLM

---

AI Gateway核心能力

┌──────────────────────────────────────────────────────┐
│                    業務服務層                          │
│   OrderService │ UserService │ ContentService         │
├──────────────────────────────────────────────────────┤
│                    AI Gateway                         │
│  ┌──────────┬──────────┬──────────┬───────────────┐  │
│  │ 路由策略  │ 限流熔斷 │ 快取管理  │ 降級策略      │  │
│  ├──────────┼──────────┼──────────┼───────────────┤  │
│  │ Prompt   │ Token    │ 審計日誌  │ 安全防護      │  │
│  │ 管理     │ 計費     │          │               │  │
│  ├──────────┴──────────┴──────────┴───────────────┤  │
│  │              流式響應代理（SSE/WebSocket）         │  │
│  ├─────────────────────────────────────────────────┤  │
│  │              多模型適配層                          │  │
│  └──┬─────────┬─────────┬─────────┬───────────────┘  │
├─────┼─────────┼─────────┼─────────┼──────────────────┤
│  OpenAI │ Claude  │ Gemini  │ 通義千問 │ DeepSeek     │
└──────────────────────────────────────────────────────┘

---

多模型路由策略

按成本/延遲/品質智慧選擇LLM

@Configuration
public class AiGatewayConfig {

    @Bean
    public ModelRouter modelRouter() {
        return ModelRouter.builder()
            .addStrategy(new CostOptimizedStrategy())
            .addStrategy(new LatencyOptimizedStrategy())
            .addStrategy(new QualityOptimizedStrategy())
            .addStrategy(new FallbackStrategy())
            .build();
    }
}

public class CostOptimizedStrategy implements RoutingStrategy {

    private static final Map<String, ModelPricing> PRICING = Map.of(
        "gpt-4o",           new ModelPricing(0.005, 0.015),
        "gpt-4o-mini",      new ModelPricing(0.00015, 0.0006),
        "claude-3.5-sonnet", new ModelPricing(0.003, 0.015),
        "deepseek-v3",      new ModelPricing(0.00027, 0.0011)
    );

    @Override
    public ModelSelection select(RoutingContext context) {
        String taskType = context.getTaskType();
        int estimatedTokens = context.getEstimatedTokens();

        return switch (taskType) {
            case "simple_qa"     -> selectModel("gpt-4o-mini", estimatedTokens);
            case "code_review"   -> selectModel("claude-3.5-sonnet", estimatedTokens);
            case "creative"      -> selectModel("gpt-4o", estimatedTokens);
            case "chinese_nlp"   -> selectModel("deepseek-v3", estimatedTokens);
            default              -> selectModel("gpt-4o", estimatedTokens);
        };
    }
}

---

Prompt模板管理與版本控制

@Service
public class PromptTemplateService {

    private final PromptTemplateRepository templateRepo;

    public PromptRenderResult render(String templateId, Map<String, String> variables) {
        PromptTemplate template = templateRepo.findLatestVersion(templateId);

        String renderedPrompt = template.getContent();
        for (Map.Entry<String, String> entry : variables.entrySet()) {
            renderedPrompt = renderedPrompt.replace("{{" + entry.getKey() + "}}", entry.getValue());
        }

        return PromptRenderResult.builder()
            .templateId(templateId)
            .version(template.getVersion())
            .renderedPrompt(renderedPrompt)
            .estimatedTokens(estimateTokens(renderedPrompt))
            .build();
    }
}

@Entity
@Table(name = "prompt_templates")
public class PromptTemplate {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String templateId;
    private Integer version;
    private String content;
    private String description;
    private Boolean isActive;

    @Column(name = "created_at")
    private LocalDateTime createdAt;
}

---

Token計費與用量追蹤

@Service
public class TokenBillingService {

    private final UsageRepository usageRepo;

    public UsageRecord recordUsage(UsageRequest request) {
        BigDecimal cost = calculateCost(
            request.getModel(),
            request.getInputTokens(),
            request.getOutputTokens()
        );

        UsageRecord record = UsageRecord.builder()
            .tenantId(request.getTenantId())
            .model(request.getModel())
            .inputTokens(request.getInputTokens())
            .outputTokens(request.getOutputTokens())
            .cost(cost)
            .promptTemplateId(request.getPromptTemplateId())
            .latencyMs(request.getLatencyMs())
            .build();

        return usageRepo.save(record);
    }

    public BillingSummary getMonthlySummary(String tenantId, YearMonth month) {
        List<UsageRecord> records = usageRepo.findByTenantIdAndMonth(tenantId, month);

        return BillingSummary.builder()
            .tenantId(tenantId)
            .month(month)
            .totalTokens(records.stream().mapToLong(r -> r.getInputTokens() + r.getOutputTokens()).sum())
            .totalCost(records.stream().map(UsageRecord::getCost).reduce(BigDecimal.ZERO, BigDecimal::add))
            .byModel(records.stream().collect(Collectors.groupingBy(UsageRecord::getModel, Collectors.summingLong(r -> r.getInputTokens() + r.getOutputTokens()))))
            .avgLatencyMs(records.stream().mapToLong(UsageRecord::getLatencyMs).average().orElse(0))
            .build();
    }
}

---

流式響應代理：SSE透傳

@RestController
@RequestMapping("/api/ai")
public class StreamingAiController {

    private final AiGatewayService gatewayService;

    @PostMapping(value = "/chat/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<ServerSentEvent<String>> streamChat(@RequestBody ChatRequest request) {
        return gatewayService.streamChat(request)
            .map(chunk -> ServerSentEvent.<String>builder()
                .id(chunk.getId())
                .event("delta")
                .data(chunk.getContent())
                .build())
            .concatWith(Flux.just(
                ServerSentEvent.<String>builder()
                    .event("done")
                    .data("[DONE]")
                    .build()
            ));
    }
}

@Service
public class AiGatewayService {

    public Flux<StreamChunk> streamChat(ChatRequest request) {
        ModelSelection selection = modelRouter.select(RoutingContext.from(request));
        LlmProvider provider = providerFactory.getProvider(selection.getModel());

        return provider.stream(request.getPrompt(), selection.getModel())
            .doOnNext(chunk -> tokenBillingService.recordAsync(
                request.getTenantId(), selection.getModel(), chunk))
            .onErrorResume(e -> fallbackProvider.stream(request.getPrompt()));
    }
}

---

Spring Cloud Gateway + AI擴展實戰

# application.yml
spring:
  cloud:
    gateway:
      routes:
        - id: openai-route
          uri: https://api.openai.com
          predicates:
            - Path=/api/ai/openai/**
          filters:
            - name: AiGateway
              args:
                provider: openai
                model: gpt-4o
                rateLimit: 100/s
                timeout: 30s

        - id: claude-route
          uri: https://api.anthropic.com
          predicates:
            - Path=/api/ai/claude/**
          filters:
            - name: AiGateway
              args:
                provider: anthropic
                model: claude-3.5-sonnet
                rateLimit: 50/s
                timeout: 60s

@Component
public class AiGatewayFilter implements GlobalFilter, Ordered {

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        String provider = exchange.getRequest().getHeaders().getFirst("X-AI-Provider");

        if (!rateLimiter.tryAcquire(provider)) {
            exchange.getResponse().setStatusCode(HttpStatus.TOO_MANY_REQUESTS);
            return exchange.getResponse().setComplete();
        }

        auditService.log(exchange.getRequest());

        return chain.filter(exchange)
            .doOnSuccess(v -> billingService.record(exchange))
            .onErrorResume(e -> fallbackService.handle(exchange, e));
    }

    @Override
    public int getOrder() {
        return -1;
    }
}

---

安全：Prompt注入防護與輸出過濾

@Service
public class AiSecurityService {

    private static final List<Pattern> INJECTION_PATTERNS = List.of(
        Pattern.compile("(?i)ignore\\s+(all\\s+)?previous\\s+instructions"),
        Pattern.compile("(?i)system\\s*:\\s*you\\s+are"),
        Pattern.compile("(?i)forget\\s+everything"),
        Pattern.compile("(?i)pretend\\s+you\\s+are")
    );

    private static final List<Pattern> SENSITIVE_PATTERNS = List.of(
        Pattern.compile("\\b\\d{16}\\b"),
        Pattern.compile("\\b\\d{17}[\\dXx]\\b"),
        Pattern.compile("[\\w.-]+@[\\w.-]+\\.\\w+")
    );

    public SecurityCheckResult checkInput(String prompt) {
        for (Pattern pattern : INJECTION_PATTERNS) {
            if (pattern.matcher(prompt).find()) {
                return SecurityCheckResult.blocked("疑似Prompt注入攻擊");
            }
        }
        return SecurityCheckResult.passed();
    }

    public String sanitizeOutput(String output) {
        String sanitized = output;
        for (Pattern pattern : SENSITIVE_PATTERNS) {
            sanitized = pattern.matcher(sanitized).replaceAll("[REDACTED]");
        }
        return sanitized;
    }
}

---

總結

1. AI Gateway是AI時代的基礎設施 — 統一接入多LLM，業務程式碼零耦合
2. 多模型路由讓成本降低40% — 按任務類型智慧選擇最優模型
3. 安全是底線 — Prompt注入防護、輸出過濾、敏感資訊脫敏缺一不可
4. Spring Cloud Gateway + AI擴展是最佳實踐 — 閘道器層統一管控，業務層無感知

AI Gateway不是可選架構，而是AI時代的必修課。越早建設，越早擺脫LLM供應商鎖定。