Inja LLM Gateway
Get started
View as Markdown

Changelog

Release history for Inja LLM Gateway

All notable changes to Inja LLM Gateway (llm-gateway) are documented here.

Format based on Keep a Changelog. Versioning follows Semantic Versioning for the public HTTP/WS surface and config schema (not only the Go module path).

Versioning policy

Bump When
MAJOR Breaking wire or config changes: removed routes, renames that break clients, required new headers, or drop-list behavior that previously preserved fields
MINOR Additive routes, optional config, new provider templates, capability flags, docs
PATCH Bug fixes, security hardening, performance, test/CI-only

Gateway Media Contract v1 (Anthropic/Google-shaped media paths, when shipped) is versioned with the gateway: additive media fields → MINOR; breaking media field renames → MAJOR. See docs/deprecation-policy.md for translation field drops.

Release process: tag vX.Y.Z → GitHub Actions builds multi-arch binaries. PRs that change the public surface should add a changelog entry under [Unreleased].

[Unreleased]

Fixed

  • cc-gateway-up after helpers install: ship scripts/gen-localhost-tls.sh with the embedded helpers and look it up under the install dir (~/.config/inja-gateway/scripts/), not only the git-checkout path examples/scripts/. Fresh helpers install layouts can create localhost TLS certs without a repo tree.
  • cc-grok in zsh: _inja_cc_normalize_providers used bash read -a, which zsh rejects (bad option: -a) and then reported empty provider set. Split on + with a portable loop; do not rely on bash-dynamic local for slot variables.
  • Claude Desktop writers: merge env into existing claude_desktop_config.json / ~/.claude/settings.json instead of replacing the file (keeps MCP, Cowork prefs, hooks). Write a 3P import JSON — current Claude Desktop routes inference via Developer → Configure Third-Party Inference, not ANTHROPIC_BASE_URL.
  • Grok 4.6 alias: grok-4.6 / grok/4.6 / inja/grok-4.6 → xai/grok-4.6 so Claude Code can /model grok-4.6 with reasoning_effort: xhigh (4.5 coerces xhigh to high).

Added

  • Docs: Moonshot / Kimi provider guide — get API keys on platform.kimi.ai, Kimi K3 (kimi-k3), regional bases, gateway curl/SDK, helpers, Kimi Code CLI /login; example aliases kimi / kimi-k3.
  • Docs: Claude app + subscriptions and Codex + subscriptions — third-party inference in the Claude desktop app; Codex CLI / ChatGPT coding via ~/.codex/config.toml.
  • claude-grok PATH wrapper: helpers install writes scripts/claude-grok; ln -sf onto PATH. Forwards all args to claude (--help does not start the gateway). Defaults Grok 4.6 + xhigh. Claude Code env uses ANTHROPIC_AUTH_TOKEN only (avoids dual-key warning).

[0.2.0] — 2026-07-24

Consumer subscription OAuth proxy (ChatGPT / Claude / SuperGrok), Claude Code / Cursor helpers, and fidelity stack for subscription backends.

Fixed

  • Subscription OAuth proxy + model list: inject Claude OAuth betas (oauth-2025-04-20, …) / X-App: cli and Codex User-Agent+Originator+Chatgpt-Account-Id for oauth.credentials providers; parse ChatGPT JWT account id on login/refresh; GET /v1/models only lists subscription aliases/catalog when the local auth store has usable credentials.

Added

  • Subscription fidelity stack: Chrome-like TLS (utls) for Anthropic/ChatGPT hosts; Claude OAuth tool-name remap + response restore; optional cloaking (Claude Code system/billing/cch/user_id, mode auto/always/never); multi-account credential pool with round-robin and 429 cooldown retry; remote model catalog refresh (INJA_GATEWAY_MODELS_URL).

  • Subscription OAuth auth CLI + store: llm-gateway auth login chatgpt|claude|grok (ChatGPT Codex PKCE, Claude setup-token, SuperGrok device-code / Grok CLI import), auth status|logout|import|env, credential store, oauth.credentials: chatgpt|claude|grok, config examples/configs/claude-code-subscriptions.yaml, package subauth.

  • Claude Code combo launcher: any provider mix (gpt, grok, gpt+grok, multi, …), Grok 4.5 + Composer 2.5 aliases, shell helpers cc-gpt / cc-grok / cc-gpt-grok / cc-run.

  • Docs site: full public guide Claude Code + subscriptions (website/src/content/docs/guides/claude-code-subscriptions.mdx); in-repo docs/claude-code-multi.md.

  • Local HTTPS for Claude Code: optional tls.cert_file / tls.key_file (or GATEWAY_TLS_CERT / GATEWAY_TLS_KEY), examples/scripts/gen-localhost-tls.sh (mkcert or openssl), background helpers cc-gateway-up / cc-gateway-down that wait for healthz and export ANTHROPIC_BASE_URL=https://127.0.0.1:8787 + NODE_EXTRA_CA_CERTS.

  • cc-gateway-logs: tail/follow ~/.local/state/inja-gateway/gateway.log from cc-gateway-up (-f, -n N, --path, --usage).

  • Request / usage process logs: each proxied request emits a human-readable usage status=… provider=… model=… tokens_in=… line; HTTP access log http status=… method=… path=… (skips /healthz /metrics). Visible in cc-gateway-logs -f.

  • Embedded shell helpers: llm-gateway helpers install (alias load-helpers) writes Claude Code / Cursor / apps helpers + subscription config from the release binary to ~/.config/inja-gateway/. helpers list|print|source|path.

  • CI coverage: overall ≥85% (includes interactive OAuth); core packages (config/proxy/ingress/egress/hooks) ≥89%.

  • Docs: all subscription / Cursor / any-app guides lead with helpers install for release binaries; document cc-gateway-logs usage lines and Cursor coexistence model names.

  • zsh helpers fix: always source claude-code-profiles.sh at load time (fixes cc-list: _inja_cc_list_profiles not found).

  • Cursor IDE + subscriptions: helpers examples/shell/cursor-helpers.sh (cursor-setup, cursor-verify), guide cursor-subscriptions — same gateway OpenAI base URL (https://127.0.0.1:8787/v1) for ChatGPT / Grok / Claude model aliases.

  • Any-app integrations: templates under examples/apps/ (Claude Desktop, Codex/ChatGPT Desktop, Continue, Cline/Roo, Aider, Windsurf, generic OpenAI/Anthropic env), shell helpers examples/shell/apps-helpers.sh (apps-setup, apps-write-claude-desktop, apps-write-claude-settings, apps-write-codex, …), docs site guide app-integrations.

  • App config profiles (backup/rollback): apps-use-gateway / apps-use-default / apps-switch store named snapshots (default, gateway, timestamped history/) under ~/.local/state/inja-gateway/app-profiles/ so Claude Desktop, ~/.claude/settings.json, and Codex can flip between llm-gateway and vendor settings.

  • Cursor coexistence model names: prefixed aliases (claude/fable-5, chatgpt/sol, grok/composer-2.5, inja/…) so Cursor built-ins (e.g. Claude Fable 5, Composer 2.5) stay in the picker next to gateway-routed custom models; cursor-models + examples/cursor/models-to-add.txt.

  • Cursor automation: cursor-apply / cursor-status / cursor-rollback write openAIBaseUrl + merge models into Cursor state.vscdb (userAddedModels); API key still one-time GUI paste (safeStorage).

  • Model alias refresh (2026-07): Claude → claude-sonnet-5 / claude-opus-4-8 / claude-haiku-4-5; ChatGPT → gpt-5.6-terra|sol|luna; xAI → grok-4.5 + grok-build-0.1 (Composer-class). Dropped retired snapshot ids from example configs.

  • Live models fan-out: GET /v1/models?live=1 merges config aliases with live upstream GET /models for openai / openai_compat / anthropic (skip on failure). Helper examples/scripts/refresh-model-catalog.sh. Maintainer rules in AGENTS.md.

  • Docs: full operator guides for OAuth, WIF, Realtime WebSocket, platform APIs, embeddings, tools, field parity, SSE, errors — in-repo docs/* and docs site MDX (website/src/content/docs/guides|reference).

  • Embeddings field parity (#148): dimensions, task_type → Gemini taskType/outputDimensionality; encoding_format validated (float only on Google translate). Docs: m6-remaining-surface.md locks regional/media/bridge policy for remaining M6 items.

  • Fidelity wave: canonical tool union (function/custom/computer/server) with OpenAI rebuild + Anthropic/Google fail-closed (#107/#161); Google thoughtSignature multi-turn (#106); expanded finishReason catalog (#156); OpenAI field parity IR (safety_identifier, verbosity, prediction, logprobs, modalities, …) (#115/#163); config.VertexBaseURL (#136); Responses WS /v1/responses/ws (#158); docs for errors/SSE/fields/Vertex/tools.

  • M6 platform proxy wave: Google /v1beta/files, /interactions, /batches, :batchGenerateContent, :asyncBatchEmbedContent; Anthropic /v1/agents|sessions|environments; OpenAI Realtime client_secrets/calls/translations, Evals, Admin organization routes, Responses compact/input_items, DELETE /v1/models, video list/delete/remix, xAI deferred completion, /v1/rerank, /v1/ocr (#110/#121/#123/#124/#128/#134/#135/#138/#142/#143/#147/#150/#197/#198).

  • WIF recipes + token_file (#164): plain access-token file TokenSource for sidecars/projected volumes; docs/wif-recipes.md (OpenAI WIF, GCP, AWS/Azure, GHA OIDC patterns).

  • Realtime / Live TLS wss dial (#105): production https/wss upstream WebSocket dial with system root CAs (TLS 1.2+), TCP keepalive, TokenSource-aware auth on upgrade; application ping/pong passed through raw. Hermetic tests via httptest.NewTLSServer.

  • Provider OAuth & non–API-key auth (#104): auth: oauth2 with YAML oauth: block (client_credentials + refresh_token TokenSources, stdlib form POST); auth: client_bearer to always forward client Bearer; auto Google SA JWT from service_account_file / GOOGLE_APPLICATION_CREDENTIALS for adc / service_account; token cache honors expires_in with single-flight refresh; one-shot 401 force-refresh retry on TokenSource modes (before client write). See docs/oauth-token-sources.md.

Changed

  • License: project relicensed from MIT to GNU Affero General Public License v3.0 (AGPL-3.0). Network use of modified versions requires offering corresponding source under the AGPL.
  • Docs cleanup: removed internal Superpowers SDD/plan trees (.superpowers/, docs/superpowers/); public contract stays in README + docs/*.md.
  • Product policy: no more wontfix / document-skip for missing endpoints. Incomplete items reopened on GitHub; full surface tracked under milestone M6 (#104–#164 + reopened stubs). README no longer marks Prometheus/health/Moonshot helpers as permanent wontfix.

Changed

  • Docs site: rebuilt on Nimbus (Astro 7) with a product landing page, filesystem sidebar, Pagefind search, and agent surfaces (llms.txt, per-page Markdown alternates).

Added

  • Opt-in Anthropic auto breakpoints: caching.auto_breakpoints injects cache_control on system/tools when translating OpenAI/Google → Anthropic (default off; client wins; X-Gateway-Cache-Auto header). docs/cache-control-policy.md.
  • Prompt caching IR (#108): Anthropic cache_control on system/content/tools preserved on Anthropic translate rebuild; OpenAI prompt_cache_key / prompt_cache_retention; Google cachedContent resource name. Cross-family still drops foreign directives. docs/cache-control-policy.md.
  • Google cachedContents CRUD proxy: /v1beta/cachedContents* for kind:google (#112 cache slice).
  • Fidelity: OpenAI service_tier request + system_fingerprint / response service_tier passthrough; never invent on Anthropic/Google translate (#51); docs/service-tier-fingerprint.md.
  • Policy: Non-function OpenAI tools error on translation path; passthrough still forwards wire tools (#49); docs/tools-policy.md.
  • Policy: Anthropic cache_control passthrough-only (Option B); translate strips breakpoints (#41); docs/cache-control-policy.md.
  • Config: optional observe_dropped_fields → response X-Gateway-Dropped-Fields + usage dropped_fields (names only) on translate (#152).
  • Moonshot helpers: POST /v1/tokenizers/estimate-token-count and GET /v1/users/me/balance thin openai_compat proxy (#89, #137).
  • Docs: DeepSeek experimental Completions/FIM operator guide (#90); docs/providers/deepseek-fim.md.
  • OpenAI Batches API proxy: POST/GET /v1/batches, GET …/{id}, POST …/{id}/cancel for openai/openai_compat (#109).
  • Ops: optional GET /v1/health/providers when health_checks.enabled (timeouts, no key logging) (#94, #153).
  • Ops: GET /metrics via prometheus/client_golang (promhttp): request counters, token counters, latency histogram, Go/process collectors (#95, #154).
  • Models: live Anthropic GET /v1/models (+ /{id}) when anthropic-version or ?live=1 (#126).
  • Proxy: header matrix docs; forward X-Client-Request-Id; expand response allowlist notes (#151).
  • OpenAI Chat storage: GET/POST/DELETE /v1/chat/completions[/{id}] for stored completions (#122).
  • OpenAI Fine-tuning: /v1/fine_tuning/jobs create/list/get/cancel/events/checkpoints proxy (#119).
  • OpenAI platform: Vector stores, Uploads, and Containers pure proxy (#113).
  • Anthropic Skills API proxy: /v1/skills (+ versions nested) (#127).
  • Anthropic MCP tunnels proxy: /v1/tunnels (#129).
  • Anthropic memory stores proxy: /v1/memory_stores (#130).
  • Google File Search stores proxy: /v1beta/fileSearchStores (#132).
  • Google Tuned models proxy: /v1beta/tunedModels (#133).
  • OpenRouter helpers: /v1/credits, /v1/key, /v1/generation (#139).
  • OpenAI Assistants/Threads/Runs legacy proxy (OpenAI-Beta forwarded) (#120).
  • Docs: Z.AI / Zhipu regional bases — intl vs CN openai_compat examples, date-stamped vendor links, curl sample (#87).
  • Docs: Qwen / DashScope regional bases — CN vs intl compatible-mode URLs, alias samples, README pointer (#88).
  • Docs: xAI Grok / Responses / Imagine — base_url, capabilities, curl + SDK samples (#91).
  • Docs: Groq STT-first routing — split chat/STT YAML, audio_transcribe, client curl (#92).
  • Docs/API: Conversations documented as not supported (501 stub); matrix row + stronger hermetic message tests (#67).
  • Decision: Conversations Option A (permanent 501; no gateway store; no pure upstream proxy) — docs/conversations-decision.md (#118).
  • Docs: Deprecation / field-drop policy acceptance locked (#103) — passthrough never drops; no Warning header; x-gateway-dropped-fields deferred to #152; hermetic doc + drop-list tests.
  • HTTP voice (TTS/STT, M4): OpenAI /v1/audio/speech|transcriptions|translations (passthrough + kind:google TTS translation); Anthropic-gateway same paths with anthropic-version (translate to OpenAI/Google); Google POST /v1beta/models/{m}:generateSpeech → Gemini generateContent AUDIO. Capability fail-closed; binary/multipart fidelity tests; usage audio_speech / audio_transcribe.
  • GET /v1/models capability flags: each catalog entry includes capabilities (chat, image_gen, video_gen, audio_speech, audio_transcribe, realtime) from provider kind defaults + YAML overrides (no upstream network).
  • Configurable max_body_bytes (default 32 MiB): oversize requests return HTTP 413 dialect-shaped errors; README limits table expanded (body, header wait, realtime, drain).
  • Multipart/media security review: docs/security-multipart-review.md linked from SECURITY.md (size limits, filenames, SSRF URI pass-through, key_hash only).
  • Ops: Prometheus /metrics and provider health shipped later under Unreleased (see Added); earlier note deferred to hooks-only.
  • Experimental Completions / DeepSeek FIM: POST /v1/completions and POST /beta/completions OpenAI-family passthrough (model rewrite + usage). /beta rewrites provider base …/v1 or host root → …/beta for DeepSeek FIM. Not multi-dialect translated.
  • Docs: SDK hermetic compatibility matrix (OpenAI/Anthropic/Google; named hermetic tests; default CI has no -tags live).
  • Docs: Moonshot/Kimi token-estimate + balance helpers tracked open (#89); DeepSeek FIM experimental under Provider notes (#90).
  • Anthropic Message Batches proxy: POST/GET /v1/messages/batches, GET …/{id}, POST …/{id}/cancel, GET …/{id}/results. Nested requests[].params.model rewrite (aliases / provider/model); provider via ?provider= / X-Provider / defaults.anthropic_dialect (kind: anthropic only). Batches/results are upstream-owned (no gateway storage).
  • Optional edge auth (edge_auth): when enabled, require Authorization: Bearer or x-api-key matching configured keys / keys_env. GET /healthz stays open. Default off.
  • Provider auth modes for Vertex-style Google hosts: auth: api_key|adc|service_account|bearer plus TokenSource interface (StaticTokenSource, CachingTokenSource) and Server.SetTokenSource for air-gapped ADC tests (no Google SDK required).
  • Forward selected client headers on upstream requests: HTTP-Referer, Referer, X-Title, OpenAI-Organization, OpenAI-Project, anthropic-beta, anthropic-version (when set by client).
  • Docs: provider notes (OpenRouter, xAI, Z.AI regions, Qwen regions, Groq STT routing), compatibility matrix, deprecation policy, Claude Code checklist, CONTRIBUTING modality guide, this changelog.
  • Conversations API stubs (/v1/conversations, /{id}, nested paths): HTTP 501 OpenAI envelope not_implemented pointing to Responses + client-side state / Files (stateless gateway decision).
  • Realtime bridge fail-closed: cross-protocol Realtime↔Live attempts return unsupported_realtime_bridge; canonical/realtime.go placeholder IR reserved for a future milestone.

Changed

  • README Auth & keys section documents edge auth vs upstream api_key_env.
  • gateway.example.yaml expands regional provider examples and edge_auth / Vertex comments.
  • README / compatibility matrix: Realtime ↔ Live bridge deferred (same-protocol passthrough only); Conversations decision documented as stub 501.

[0.1.0] — prior

Initial public surface (chat OpenAI/Anthropic/Google, image/video OpenAI-compat passthrough, hooks, Docker/K8s). See git history for pre-changelog releases.

Type to search…

↑↓ navigate↵ selectEsc close