[DEPRECATED] LiteLLM Gateway — Routing Table (Phase 1)

Status: OBSOLETE since Jul 2026. Tier×Effort routing was replaced in Phase 3 (LiteLLM Proxy Server). Current reference is fitalabs-gateway-modelos. The original litellm-gateway.md documents Phase 1 (custom FastAPI) which was decommissioned.

Multi-provider proxy at localhost:4000. Claude Code sends standard names (claude-haiku-4-5, claude-sonnet-4-6, claude-opus-4-8) and the gateway detects the tier by prefix.

Effort Detection

Effort is detected from the thinking.budget_tokens field in the Anthropic request body:

Budget tokens	Effort
0 – 2047	low
2048 – 8191	medium
≥ 8192	high

No thinking block → low (default).

Main Table

🔵 Haiku

Effort	Model	Provider	Cost
low	mimo-auto → fallback mimo-v2.5	MiMo Auto 🆓 → Token Plan 💰	¥0 → ¥99/month
medium	mimo-v2.5	MiMo Token Plan 💰	¥99/month
high	deepseek-v4-flash	DeepSeek API 🧠	$0.14/$0.28 per 1M tok

🟡 Sonnet

Effort	Model	Provider	Cost
low	mimo-v2.5	MiMo Token Plan 💰	¥99/month
medium	mimo-v2.5[1m]	MiMo Token Plan 💰	¥99/month
high	deepseek-v4-flash[1m]	DeepSeek API 🧠	$0.14/$0.28 per 1M tok

🔴 Opus

Effort	Model	Provider	Cost
low	mimo-v2.5-pro	MiMo Token Plan 💰	¥99/month
medium	mimo-v2.5-pro[1m]	MiMo Token Plan 💰	¥99/month
high	deepseek-v4-pro[1m]	DeepSeek API 🧠🐘	$0.435/$0.87 per 1M tok

🖼️ Images (Multimodal)

Always: mimo-auto 🆓 → fallback mimo-v2.5 💰

• Auto-detected: any content block with type: "image"
• Ignores effort level — image always goes to the omni model

🌐 Web Search

Tier + Effort	Executor
low/medium (MiMo)	MiMo native web_search tool → SearXNG local :8888 (fallback)
high (DeepSeek)	DeepSeek native (automatic)

🛡️ Fallbacks

Primary	Fallback
mimo-auto 🆓	mimo-v2.5 💰 (on 401/403, tries JWT refresh first)
Any backend	502 error with clear message (missing API key)

Usage

# Terminal 1: start the gateway
cd ~/.claude/infra/litellm-gateway
uv run uvicorn gateway:app --host 127.0.0.1 --port 4000

# Claude Code settings.json:
# "ANTHROPIC_BASE_URL": "http://localhost:4000"
# "ANTHROPIC_AUTH_TOKEN": "sk-gateway-local"
# (no ANTHROPIC_DEFAULT_*_MODEL)

Current State (2026-06-23)

• 100/100 tests passing (63 unit + 16 integration + 11 E2E + 10 integrity)
• MiMo Auto free tier: disabled (rate-limited by Xiaomi — 441 risk_control)
• LokiProxy: tested and removed (MiMo blocks proxy IPs)
• Caddy HTTPS: running at :8443 for Claude Desktop 3p
• Voice mode: Claude Desktop voice is Anthropic-only; TTS/ASR endpoints on gateway planned

Cost Strategy

• ¥0 on MiMo Auto (free) for Haiku low and images
• ¥99/month fixed on Token Plan for all low/medium (Haiku, Sonnet, Opus)
• Pay-per-use DeepSeek only when thinking.budget_tokens ≥ 8192 (high effort)
• SearXNG local (Docker) as search fallback — zero cost, no limits

Estimated Savings

With moderate usage (~100M input + 20M output/month):

• All on DeepSeek V4 Pro: ~R$ 313/month
• With MiMo Token Plan + DeepSeek only on high: ~R$ 75/month (fixed plan)
• Savings: ~R$ 238/month 💰