Compare commits

..

114 Commits

Author SHA1 Message Date
airkjw 4e96fc3813 docs: add v6.10.4 update history
v6.9.38 → v6.10.4 (69 commits): WebSocket compact handling,
session affinity via X-Amp-Thread-Id, Codex reasoning/image
improvements, GPT-5.5 model, optional OpenAI-compat disable.
Updated without re-auth.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 13:27:37 +09:00
JOUNGWOOK KWON a79b78d64a docs: add v6.9.38 update history
v6.9.38 introduces protocol multiplexer + Redis queue and IP ban
on repeated management/Redis AUTH failures. Updated without re-auth.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 18:22:04 +09:00
JOUNGWOOK KWON 841e16536d docs: add update history (v6.9.7 → v6.9.34)
Auth file format unchanged so no re-authentication needed.
Added update procedure section to DOCKER_DEPLOY.md for future reference.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 00:20:44 +09:00
JOUNGWOOK KWON f93dde43df docs: add API usage guide with multi-language code examples
curl, Python(Anthropic/OpenAI SDK), Node.js(Anthropic/OpenAI SDK),
Claude Code CLI 등 다양한 호출 방법 정리

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 21:23:22 +09:00
JOUNGWOOK KWON 84a2874c7b docs: add Docker deployment and reverse proxy setup guides
- DOCKER_DEPLOY.md: NAS Docker 배포 전체 가이드 (컨테이너 설정, config.yaml, 포트 등)
- REVERSE_PROXY_SETUP.md: cliproxy.gru.farm HTTPS/역방향 프록시 설정 가이드

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 21:20:55 +09:00
Luis Pater c10f8ae2e2 Fixed: #2420
docker-image / docker_amd64 (push) Has been cancelled
docker-image / docker_arm64 (push) Has been cancelled
goreleaser / goreleaser (push) Has been cancelled
docker-image / docker_manifest (push) Has been cancelled
docs(readme): remove ProxyPal section from all README translations
2026-03-31 07:23:02 +08:00
Luis Pater 17363edf25 fix(auth): skip downtime for request-scoped 404 errors in model state management 2026-03-30 22:22:42 +08:00
Luis Pater 486cd4c343 Merge pull request #2409 from sususu98/fix/tool-use-pairing-break
fix(antigravity): reorder model parts to prevent tool_use↔tool_result pairing breakage
2026-03-30 16:59:46 +08:00
sususu98 25feceb783 fix(antigravity): reorder model parts to prevent tool_use↔tool_result pairing breakage
When a Claude assistant message contains [text, tool_use, text], the
Antigravity API internally splits the model message at functionCall
boundaries, creating an extra assistant turn between tool_use and the
following tool_result. Claude then rejects with:

  tool_use ids were found without tool_result blocks immediately after

Fix: extend the existing 2-way part reordering (thinking-first) to a
3-way partition: thinking → regular → functionCall. This ensures
functionCall parts are always last, so Antigravity's split cannot
insert an extra assistant turn before the user's tool_result.

Fixes #989
2026-03-30 15:09:33 +08:00
Luis Pater d26752250d Merge pull request #2403 from CharTyr/clean-pr
fix(amp): 修复Amp CLI 集成 缺失/无效 signature 导致的 TUI 崩溃与上游 400 问题
2026-03-30 12:54:15 +08:00
CharTyr b15453c369 fix(amp): address PR review - stream thinking suppression, SSE detection, test init
- Call suppressAmpThinking in rewriteStreamEvent for streaming path
- Handle nil return from suppressAmpThinking to skip suppressed events
- Narrow looksLikeSSEChunk to line-prefix detection (HasPrefix vs Contains)
- Initialize suppressedContentBlock map in test
2026-03-30 00:42:04 -04:00
CharTyr 04ba8c8bc3 feat(amp): sanitize signatures and handle stream suppression for Amp compatibility 2026-03-29 22:23:18 -04:00
Luis Pater 6570692291 Merge pull request #2400 from router-for-me/revert-2374-codex-cache-clean
docker-image / docker_amd64 (push) Has been cancelled
docker-image / docker_arm64 (push) Has been cancelled
goreleaser / goreleaser (push) Has been cancelled
docker-image / docker_manifest (push) Has been cancelled
Revert "fix(codex): restore prompt cache continuity for Codex requests"
2026-03-29 22:19:39 +08:00
Luis Pater 13aa5b3375 Revert "fix(codex): restore prompt cache continuity for Codex requests" 2026-03-29 22:18:14 +08:00
Luis Pater 6d8de0ade4 feat(auth): implement weighted provider rotation for improved scheduling fairness 2026-03-29 13:49:01 +08:00
Luis Pater 1587ff5e74 Merge pull request #2389 from router-for-me/claude
fix(claude): add default max_tokens for models
2026-03-29 13:03:20 +08:00
hkfires f033d3a6df fix(claude): enhance ensureModelMaxTokens to use registered max_completion_tokens and fallback to default 2026-03-29 13:00:43 +08:00
hkfires 145e0e0b5d fix(claude): add default max_tokens for models 2026-03-29 12:46:00 +08:00
Luis Pater 9b7d7021af docs(readme): update LingtrueAPI link in all README translations
docker-image / docker_amd64 (push) Has been cancelled
docker-image / docker_arm64 (push) Has been cancelled
goreleaser / goreleaser (push) Has been cancelled
docker-image / docker_manifest (push) Has been cancelled
2026-03-29 12:30:24 +08:00
Luis Pater e41c22ef44 docs(readme): add LingtrueAPI sponsorship details to all README translations 2026-03-29 12:23:37 +08:00
Luis Pater 55271403fb Merge pull request #2374 from VooDisss/codex-cache-clean
fix(codex): restore prompt cache continuity for Codex requests
2026-03-28 21:16:51 +08:00
Luis Pater 36fba66619 Merge pull request #2371 from RaviTharuma/docs/provider-specific-routes
docs: clarify provider-specific routing for aliased models
2026-03-28 21:11:29 +08:00
Luis Pater b9b127a7ea Merge pull request #2347 from edlsh/fix/codex-strip-stream-options
fix(codex): strip stream_options from Responses API requests
2026-03-28 21:03:01 +08:00
Luis Pater 2741e7b7b3 Merge pull request #2346 from pjpjq/codex/fix-codex-capacity-retry
fix(codex): Treat Codex capacity errors as retryable
2026-03-28 21:00:50 +08:00
Luis Pater 1767a56d4f Merge pull request #2343 from kongkk233/fix/proxy-transport-defaults
Preserve default transport settings for proxy clients
2026-03-28 20:58:24 +08:00
Luis Pater 779e6c2d2f Merge pull request #2231 from 7RPH/fix/responses-stream-multi-tool-calls
fix: preserve separate streamed tool calls in Responses API
2026-03-28 20:53:19 +08:00
Luis Pater 73c831747b Merge pull request #2133 from DragonFSKY/fix/2061-stale-modelstates
fix(auth): prevent stale runtime state inheritance from disabled auth entries
2026-03-28 20:50:57 +08:00
Luis Pater 10b824fcac fix(security): validate auth file names to prevent unsafe input
docker-image / docker_amd64 (push) Has been cancelled
docker-image / docker_arm64 (push) Has been cancelled
goreleaser / goreleaser (push) Has been cancelled
docker-image / docker_manifest (push) Has been cancelled
2026-03-28 04:48:23 +08:00
VooDisss e5d3541b5a refactor(codex): remove stale affinity cleanup leftovers
Drop the last affinity-related executor artifacts so the PR stays focused on the minimal Codex continuity fix set: stable prompt cache identity, stable session_id, and the executor-only behavior that was validated to restore cache reads.
2026-03-27 20:40:26 +02:00
VooDisss 79755e76ea refactor(pr): remove forbidden translator changes
Drop the chat-completions translator edits from this PR so the branch complies with the repository policy that forbids pull-request changes under internal/translator. The remaining PR stays focused on the executor-level Codex continuity fix that was validated to restore cache reuse.
2026-03-27 19:34:13 +02:00
VooDisss 35f158d526 refactor(pr): narrow Codex cache fix scope
Remove the experimental auth-affinity routing changes from this PR so it stays focused on the validated Codex continuity fix. This keeps the prompt-cache repair while avoiding unrelated routing-policy concerns such as provider/model affinity scope, lifecycle cleanup, and hard-pin fallback semantics.
2026-03-27 19:06:34 +02:00
VooDisss 6962e09dd9 fix(auth): scope affinity by provider
Keep sticky auth affinity limited to matching providers and stop persisting execution-session IDs as long-lived affinity keys so provider switching and normal streaming traffic do not create incorrect pins or stale affinity state.
2026-03-27 18:52:58 +02:00
VooDisss 4c4cbd44da fix(auth): avoid leaking or over-persisting affinity keys
Stop using one-shot idempotency keys as long-lived auth-affinity identifiers and remove raw affinity-key values from debug logs so sticky routing keeps its continuity benefits without creating avoidable memory growth or credential exposure risks.
2026-03-27 18:34:51 +02:00
VooDisss 26eca8b6ba fix(codex): preserve continuity and safe affinity fallback
Restore Claude continuity after the continuity refactor, keep auth-affinity keys out of upstream Codex session identifiers, and only persist affinity after successful execution so retries can still rotate to healthy credentials when the first auth fails.
2026-03-27 18:27:33 +02:00
VooDisss 62b17f40a1 refactor(codex): align continuity helpers with review feedback
Align websocket continuity resolution with the HTTP Codex path, make auth-affinity principal keys use a stable string representation, and extract small helpers that remove duplicated continuity and affinity logic without changing the validated cache-hit behavior.
2026-03-27 18:11:57 +02:00
VooDisss 511b8a992e fix(codex): restore prompt cache continuity for Codex requests
Prompt caching on Codex was not reliably reusable through the proxy because repeated chat-completions requests could reach the upstream without the same continuity envelope. In practice this showed up most clearly with OpenCode, where cache reads worked in the reference client but not through CLIProxyAPI, although the root cause is broader than OpenCode itself.

The proxy was breaking continuity in several ways: executor-layer Codex request preparation stripped prompt_cache_retention, chat-completions translation did not preserve that field, continuity headers used a different shape than the working client behavior, and OpenAI-style Codex requests could be sent without a stable prompt_cache_key. When that happened, session_id fell back to a fresh random value per request, so upstream Codex treated repeated requests as unrelated turns instead of as part of the same cacheable context.

This change fixes that by preserving caller-provided prompt_cache_retention on Codex execution paths, preserving prompt_cache_retention when translating OpenAI chat-completions requests to Codex, aligning Codex continuity headers to session_id, and introducing an explicit Codex continuity policy that derives a stable continuity key from the best available signal. The resolution order prefers an explicit prompt_cache_key, then execution session metadata, then an explicit idempotency key, then stable request-affinity metadata, then a stable client-principal hash, and finally a stable auth-ID hash when no better continuity signal exists.

The same continuity key is applied to both prompt_cache_key in the request body and session_id in the request headers so repeated requests reuse the same upstream cache/session identity. The auth manager also keeps auth selection sticky for repeated request sequences, preventing otherwise-equivalent Codex requests from drifting across different upstream auth contexts and accidentally breaking cache reuse.

To keep the implementation maintainable, the continuity resolution and diagnostics are centralized in a dedicated Codex continuity helper instead of being scattered across executor flow code. Regression coverage now verifies retention preservation, continuity-key precedence, stable auth-ID fallback, websocket parity, translator preservation, and auth-affinity behavior. Manual validation confirmed prompt cache reads now occur through CLIProxyAPI when using Codex via OpenCode, and the fix should also benefit other clients that rely on stable repeated Codex request continuity.
2026-03-27 17:49:29 +02:00
Luis Pater 7dccc7ba2f docs(readme): remove redundant whitespace in BmoPlus sponsorship section of Chinese README 2026-03-27 20:52:14 +08:00
Luis Pater 70c90687fd docs(readme): fix formatting in BmoPlus sponsorship section of Chinese README 2026-03-27 20:49:43 +08:00
Luis Pater 8144ffd5c8 Merge pull request #2370 from B3o/add-bmoplus-sponsor
docs: add BmoPlus sponsorship banners to READMEs
2026-03-27 20:48:22 +08:00
Ravi Tharuma 0ab977c236 docs: clarify provider path limitations 2026-03-27 11:13:08 +01:00
Ravi Tharuma 224f0de353 docs: neutralize provider-specific path wording 2026-03-27 11:11:06 +01:00
B3o 6b45d311ec add BmoPlus sponsorship banners to READMEs 2026-03-27 18:01:35 +08:00
Ravi Tharuma d54de441d3 docs: clarify provider-specific routing for aliased models 2026-03-27 10:53:09 +01:00
白金 1821bf7051 docs: add BmoPlus sponsorship banners to READMEs 2026-03-27 17:39:29 +08:00
Luis Pater d42b5d4e78 docs(readme): update QQ group information in Chinese README
docker-image / docker_amd64 (push) Has been cancelled
docker-image / docker_arm64 (push) Has been cancelled
goreleaser / goreleaser (push) Has been cancelled
docker-image / docker_manifest (push) Has been cancelled
2026-03-27 11:46:21 +08:00
edlsh 754f3bcbc3 fix(codex): strip stream_options from Responses API requests
The Codex/OpenAI Responses API does not support the stream_options
parameter. When clients (e.g. Amp CLI) include stream_options in their
requests, CLIProxyAPI forwards it as-is, causing a 400 error:

  {"detail":"Unsupported parameter: stream_options"}

Strip stream_options alongside the other unsupported parameters
(previous_response_id, prompt_cache_retention, safety_identifier)
in Execute, ExecuteStream, and CountTokens.
2026-03-25 11:58:36 -04:00
pjpj 36973d4a6f Handle Codex capacity errors as retryable 2026-03-25 23:25:31 +08:00
kwz c89d19b300 Preserve default transport settings for proxy clients 2026-03-25 15:33:09 +08:00
Luis Pater 1e6bc81cfd refactor(config): replace auto-update-panel with disable-auto-update-panel for clarity 2026-03-25 10:31:44 +08:00
Luis Pater 1a149475e0 Merge pull request #2293 from Xvvln/fix/management-asset-security
fix(security): harden management panel asset updater
2026-03-25 10:22:49 +08:00
Luis Pater e5166841db Merge pull request #2310 from shellus/fix/claude-openai-system-top-level
fix: preserve OpenAI system messages as Claude top-level system
2026-03-25 10:21:18 +08:00
Luis Pater bb9b2d1758 Merge pull request #2320 from cikichen/build/freebsd-support
build: add freebsd support for releases
2026-03-25 10:12:35 +08:00
Luis Pater 76c064c729 Merge pull request #2335 from router-for-me/auth
docker-image / docker_amd64 (push) Has been cancelled
docker-image / docker_arm64 (push) Has been cancelled
goreleaser / goreleaser (push) Has been cancelled
docker-image / docker_manifest (push) Has been cancelled
Support batch upload and delete for auth files
2026-03-25 09:34:44 +08:00
Luis Pater d2f652f436 Merge pull request #2333 from router-for-me/codex
feat(codex): pass through codex client identity headers
2026-03-25 09:34:09 +08:00
Luis Pater 6a452a54d5 Merge pull request #2316 from router-for-me/openai
Add per-model thinking support for OpenAI compatibility
2026-03-25 09:31:28 +08:00
hkfires 9e5693e74f feat(api): support batch auth file upload and delete 2026-03-25 09:20:17 +08:00
hkfires 528b1a2307 feat(codex): pass through codex client identity headers 2026-03-25 08:48:18 +08:00
Luis Pater 0cc978ec1d Merge pull request #2297 from router-for-me/readme
docs(readme): update japanese documentation links
2026-03-25 03:11:24 +08:00
simon d312422ab4 build: add freebsd support to releases 2026-03-24 16:49:04 +08:00
hkfires fee736933b feat(openai-compat): add per-model thinking support 2026-03-24 14:21:12 +08:00
GeJiaXiang 09c92aa0b5 fix: keep a fallback turn for system-only Claude inputs 2026-03-24 13:54:25 +08:00
GeJiaXiang 8c67b3ae64 test: verify remaining user message after system merge 2026-03-24 13:47:52 +08:00
GeJiaXiang 000e4ceb4e fix: map OpenAI system messages to Claude top-level system 2026-03-24 13:42:33 +08:00
hkfires 5c99846ecf docs(readme): update japanese documentation links 2026-03-24 09:47:01 +08:00
trph cc32f5ff61 fix: unify Responses output indexes for streamed items 2026-03-24 08:59:09 +08:00
trph fbff68b9e0 fix: preserve choice-aware output indexes for streamed tool calls 2026-03-24 08:54:43 +08:00
trph 7e1a543b79 fix: preserve separate streamed tool calls in Responses API 2026-03-24 08:51:15 +08:00
Luis Pater d475aaba96 Fixed: #2274
fix(translator): omit null content fields in Codex OpenAI tool call responses
2026-03-24 01:00:57 +08:00
Luis Pater 96f55570f7 Merge pull request #2282 from eltociear/add-ja-doc
docker-image / docker_amd64 (push) Has been cancelled
docker-image / docker_arm64 (push) Has been cancelled
goreleaser / goreleaser (push) Has been cancelled
docker-image / docker_manifest (push) Has been cancelled
docs: add Japanese README
2026-03-24 00:40:58 +08:00
Luis Pater 0906aeca87 Merge pull request #2254 from clcc2019/main
refactor: streamline usage reporting by consolidating record publishi…
2026-03-24 00:39:31 +08:00
Xvvln 7333619f15 fix: reject oversized downloads instead of truncating; warn on unverified fallback
- Read maxAssetDownloadSize+1 bytes and error if exceeded, preventing
  silent truncation that could write a broken management.html to disk
- Log explicit warning when fallback URL is used without digest
  verification, so users are aware of the reduced security guarantee
2026-03-24 00:27:44 +08:00
Luis Pater 97c0487add Merge pull request #2223 from cnrpman/fix/codex-responses-web-search-preview-compat
fix: normalize web_search_preview for codex responses
2026-03-24 00:25:37 +08:00
DragonFSKY 74b862d8b8 test(cliproxy): cover delete re-add stale state flow 2026-03-24 00:21:04 +08:00
Xvvln 2db8df8e38 fix(security): harden management panel asset updater
- Abort update when SHA256 digest mismatch is detected instead of
  logging a warning and proceeding (prevents MITM asset replacement)
- Cap asset download size to 10 MB via io.LimitReader (defense-in-depth
  against OOM from oversized responses)
- Add `auto-update-panel` config option (default: false) to make the
  periodic background updater opt-in; the panel is still downloaded
  on first access when missing, but no longer silently auto-updated
  every 3 hours unless explicitly enabled
2026-03-24 00:10:04 +08:00
Luis Pater a576088d5f Merge pull request #2222 from kaitranntt/kai/fix/758-openai-proxy-alternating-model-support
fix: fall back on model support errors during auth rotation
2026-03-24 00:03:28 +08:00
Luis Pater 66ff916838 Merge pull request #2220 from xulongwu4/main
fix: normalize model name in TranslateRequest fallback to prevent prefix leak
2026-03-23 23:56:15 +08:00
Luis Pater 7b0453074e Merge pull request #2219 from beck-8/fix/context-done-race
fix: avoid data race when watching request cancellation
2026-03-23 22:57:21 +08:00
Luis Pater a000eb523d Merge pull request #2213 from TTTPOB/ua-fix
feat(claude): stabilize device fingerprint across mixed Claude Code and cloaked clients
2026-03-23 22:53:51 +08:00
Luis Pater 18a4fedc7f Merge pull request #2126 from ailuntz/fix/watcher-auth-cache-memory
perf(watcher): reduce auth cache memory
2026-03-23 22:47:34 +08:00
Luis Pater 5d6cdccda0 Merge pull request #2268 from sususu98/fix/sanitize-tool-names
fix(translator): sanitize tool names for Gemini function_declarations compatibility
2026-03-23 21:42:22 +08:00
Luis Pater 1b7f4ac3e1 Merge pull request #2252 from sususu98/fix/antigravity-empty-thought-text
fix(antigravity): always include text field in thought parts to prevent Google 500
2026-03-23 21:41:25 +08:00
Luis Pater afc1a5b814 Fixed: #2281
refactor(claude): centralize usage token calculation logic and add tests for cached token handling
2026-03-23 21:30:03 +08:00
Ikko Eltociear Ashimine 7ed38db54f docs: update README_JA.md
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-23 16:57:43 +09:00
Ikko Eltociear Ashimine 28c10f4e69 docs: update README_JA.md
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-23 16:57:32 +09:00
Ikko Eltociear Ashimine 6e12441a3b Update README_JA.md
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-23 16:57:19 +09:00
Ikko Ashimine 65c439c18d docs: add Japanese README 2026-03-23 15:23:18 +09:00
dslife2025 0ed2d16596 Merge branch 'router-for-me:main' into main 2026-03-23 09:50:43 +08:00
Supra4E8C db335ac616 Merge pull request #2269 from router-for-me/auth-fix
fix(auth): ensure absolute paths for auth file handling
2026-03-22 22:53:44 +08:00
sususu98 e8bb350467 fix: extend tool name sanitization to all remaining Gemini-bound translators
Apply SanitizeFunctionName on request and RestoreSanitizedToolName on
response for: gemini/claude, gemini/openai/chat-completions,
gemini/openai/responses, antigravity/openai/chat-completions,
gemini-cli/openai/chat-completions.

Also update SanitizedToolNameMap to handle OpenAI format
(tools[].function.name) in addition to Claude format (tools[].name).
2026-03-22 14:06:46 +08:00
Supra4E8C 5331d51f27 fix(auth): ensure absolute paths for auth file handling 2026-03-22 13:58:16 +08:00
sususu98 755ca75879 fix: address review feedback - init ToolNameMap eagerly, log collisions, add collision test 2026-03-22 13:24:03 +08:00
sususu98 2398ebad55 fix(translator): sanitize tool names for Gemini function_declarations compatibility
Claude Code and MCP clients may send tool names containing characters
invalid for Gemini's function_declarations (e.g. '/', '@', spaces).
Sanitize on request via SanitizeFunctionName and restore original names
on response for both antigravity/claude and gemini-cli/claude translators.
2026-03-22 13:10:53 +08:00
clcc2019 c1bf298216 refactor: streamline usage reporting by consolidating record publishing logic
- Introduced a new method `buildRecord` in `usageReporter` to encapsulate record creation, improving code readability and maintainability.
- Added latency tracking to usage records, ensuring accurate reporting of request latencies.
- Updated tests to validate the inclusion of latency in usage records and ensure proper functionality of the new reporting structure.
2026-03-20 19:44:26 +08:00
sususu e005208d76 fix(antigravity): always include text field in thought parts to prevent Google 500
When Claude sends redacted thinking with empty text, the translator
was omitting the "text" field from thought parts. Google Antigravity
API requires this field, causing 500 "Unknown Error" responses.

Verified: 129/129 error logs with empty thought → 500, 0/97 success
logs had empty thought. After fix: 0 new "Unknown Error" 500s.
2026-03-20 18:59:25 +08:00
Junyi Du d1df70d02f chore: add codex builtin tool normalization logging 2026-03-20 14:08:37 +08:00
tpob 52c1fa025e fix(claude): learn official fingerprints after custom baselines 2026-03-19 13:59:41 +08:00
tpob 680105f84d fix(claude): refresh cached fingerprint after baseline upgrades 2026-03-19 13:28:58 +08:00
tpob f7069e9548 fix(claude): pin stabilized OS arch to baseline 2026-03-19 13:07:16 +08:00
Junyi Du 793840cdb4 fix: cover dated and nested codex web search aliases 2026-03-19 03:41:12 +08:00
Junyi Du 8f421de532 fix: handle sjson errors in codex tool normalization 2026-03-19 03:36:06 +08:00
Junyi Du be2dd60ee7 fix: normalize web_search_preview for codex responses 2026-03-19 03:23:14 +08:00
Tam Nhu Tran ea3e0b713e fix: harden pooled model-support fallback state 2026-03-18 13:19:20 -04:00
tpob 8179d5a8a4 fix(claude): avoid racy fingerprint downgrades 2026-03-19 01:03:41 +08:00
tpob 6fa7abe434 fix(claude): keep configured baseline above older fingerprints 2026-03-19 01:02:04 +08:00
Tam Nhu Tran 5135c22cd6 fix: fall back on model support errors during auth rotation 2026-03-18 12:43:45 -04:00
Longwu Ou 1e27990561 address PR review: log sjson error and add unit tests
- Log a warning instead of silently ignoring sjson.SetBytes errors in the TranslateRequest fallback path
  - Add registry_test.go with tests covering the fallback model normalization and verifying registered transforms take precedence
2026-03-18 12:43:40 -04:00
Longwu Ou e1e9fc43c1 fix: normalize model name in TranslateRequest fallback to prevent prefix leak
When no request translator is registered for a format pair (e.g.
        openai-response → openai-response), TranslateRequest returned the raw
        payload unchanged. This caused client-side model prefixes (e.g.
        "copilot/gpt-5-mini") to leak into upstream requests, resulting in
        "The requested model is not supported" errors from providers.

        The fallback path now updates the "model" field in the payload to
        match the resolved model name before returning.
2026-03-18 12:30:22 -04:00
beck-8 b2921518ac fix: avoid data race when watching request cancellation 2026-03-19 00:15:52 +08:00
tpob dd64adbeeb fix(claude): preserve legacy user agent overrides 2026-03-19 00:03:09 +08:00
tpob 616d41c06a fix(claude): restore legacy runtime OS arch fallback 2026-03-19 00:01:50 +08:00
tpob e0e337aeb9 feat(claude): add switch for device profile stabilization 2026-03-18 19:31:59 +08:00
tpob d52839fced fix: stabilize claude device fingerprint 2026-03-18 18:46:54 +08:00
DragonFSKY 5c817a9b42 fix(auth): prevent stale ModelStates inheritance from disabled auth entries
When an auth file is deleted and re-created with the same path/ID, the
new auth could inherit stale ModelStates (cooldown/backoff) from the
previously disabled entry, preventing it from being routed.

Gate runtime state inheritance (ModelStates, LastRefreshedAt,
NextRefreshAfter) on both existing and incoming auth being non-disabled
in Manager.Update and Service.applyCoreAuthAddOrUpdate.

Closes #2061
2026-03-14 23:46:23 +08:00
ailuntz c3762328a5 perf(watcher): reduce auth cache memory 2026-03-10 16:27:10 +08:00
76 changed files with 6118 additions and 592 deletions
+1
View File
@@ -8,6 +8,7 @@ builds:
- linux
- windows
- darwin
- freebsd
goarch:
- amd64
- arm64
+343
View File
@@ -0,0 +1,343 @@
# CLIProxyAPI 호출 가이드
## 접속 정보
| 항목 | 값 |
|------|-----|
| 외부 URL | `https://cliproxy.gru.farm` |
| 내부 URL | `http://192.168.0.17:8317` |
| API 키 | `Jinie4eva!` |
| 인증 방식 | `Authorization: Bearer <API키>` |
## 엔드포인트
| 용도 | 경로 |
|------|------|
| Claude 네이티브 (권장) | `/api/provider/claude/v1/messages` |
| OpenAI 호환 | `/v1/chat/completions` |
| 모델 목록 | `/v1/models` |
## 사용 가능한 모델
| 모델 ID | 설명 |
|---------|------|
| `claude-sonnet-4-6` | Claude Sonnet 4.6 (최신, 권장) |
| `claude-opus-4-6` | Claude Opus 4.6 (최고 성능) |
| `claude-sonnet-4-5-20250929` | Claude Sonnet 4.5 |
| `claude-opus-4-5-20251101` | Claude Opus 4.5 |
| `claude-haiku-4-5-20251001` | Claude Haiku 4.5 (경량/빠름) |
| `claude-sonnet-4-20250514` | Claude Sonnet 4 |
| `claude-opus-4-20250514` | Claude Opus 4 |
| `claude-3-7-sonnet-20250219` | Claude 3.7 Sonnet |
| `claude-3-5-haiku-20241022` | Claude 3.5 Haiku |
---
## 1. curl
### 기본 호출
```bash
curl -X POST https://cliproxy.gru.farm/api/provider/claude/v1/messages \
-H "Authorization: Bearer Jinie4eva!" \
-H "anthropic-version: 2023-06-01" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-sonnet-4-6",
"max_tokens": 1024,
"messages": [
{"role": "user", "content": "안녕! 간단히 소개해줘"}
]
}'
```
### 스트리밍
```bash
curl -X POST https://cliproxy.gru.farm/api/provider/claude/v1/messages \
-H "Authorization: Bearer Jinie4eva!" \
-H "anthropic-version: 2023-06-01" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-sonnet-4-6",
"max_tokens": 1024,
"stream": true,
"messages": [
{"role": "user", "content": "안녕!"}
]
}'
```
### 모델 목록 조회
```bash
curl https://cliproxy.gru.farm/v1/models \
-H "Authorization: Bearer Jinie4eva!"
```
---
## 2. Python — Anthropic SDK
### 설치
```bash
pip install anthropic
```
### 기본 호출
```python
from anthropic import Anthropic
client = Anthropic(
base_url="https://cliproxy.gru.farm/api/provider/claude",
api_key="Jinie4eva!"
)
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=1024,
messages=[
{"role": "user", "content": "안녕! 간단히 소개해줘"}
]
)
print(response.content[0].text)
```
### 스트리밍
```python
from anthropic import Anthropic
client = Anthropic(
base_url="https://cliproxy.gru.farm/api/provider/claude",
api_key="Jinie4eva!"
)
with client.messages.stream(
model="claude-sonnet-4-6",
max_tokens=1024,
messages=[
{"role": "user", "content": "안녕! 간단히 소개해줘"}
]
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
```
### 시스템 프롬프트 + 멀티턴
```python
from anthropic import Anthropic
client = Anthropic(
base_url="https://cliproxy.gru.farm/api/provider/claude",
api_key="Jinie4eva!"
)
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=1024,
system="당신은 친절한 한국어 AI 어시스턴트입니다.",
messages=[
{"role": "user", "content": "파이썬이 뭐야?"},
{"role": "assistant", "content": "파이썬은 프로그래밍 언어입니다."},
{"role": "user", "content": "그럼 자바스크립트는?"}
]
)
print(response.content[0].text)
```
---
## 3. Python — OpenAI SDK (호환 모드)
### 설치
```bash
pip install openai
```
### 기본 호출
```python
from openai import OpenAI
client = OpenAI(
base_url="https://cliproxy.gru.farm/v1",
api_key="Jinie4eva!"
)
response = client.chat.completions.create(
model="claude-sonnet-4-6",
messages=[
{"role": "user", "content": "안녕!"}
]
)
print(response.choices[0].message.content)
```
### 스트리밍
```python
from openai import OpenAI
client = OpenAI(
base_url="https://cliproxy.gru.farm/v1",
api_key="Jinie4eva!"
)
stream = client.chat.completions.create(
model="claude-sonnet-4-6",
messages=[{"role": "user", "content": "안녕!"}],
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
```
---
## 4. Node.js — Anthropic SDK
### 설치
```bash
npm install @anthropic-ai/sdk
```
### 기본 호출
```javascript
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
baseURL: "https://cliproxy.gru.farm/api/provider/claude",
apiKey: "Jinie4eva!",
});
const response = await client.messages.create({
model: "claude-sonnet-4-6",
max_tokens: 1024,
messages: [{ role: "user", content: "안녕!" }],
});
console.log(response.content[0].text);
```
### 스트리밍
```javascript
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
baseURL: "https://cliproxy.gru.farm/api/provider/claude",
apiKey: "Jinie4eva!",
});
const stream = client.messages.stream({
model: "claude-sonnet-4-6",
max_tokens: 1024,
messages: [{ role: "user", content: "안녕!" }],
});
for await (const chunk of stream) {
if (
chunk.type === "content_block_delta" &&
chunk.delta.type === "text_delta"
) {
process.stdout.write(chunk.delta.text);
}
}
```
---
## 5. Node.js — OpenAI SDK (호환 모드)
### 설치
```bash
npm install openai
```
### 기본 호출
```javascript
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://cliproxy.gru.farm/v1",
apiKey: "Jinie4eva!",
});
const response = await client.chat.completions.create({
model: "claude-sonnet-4-6",
messages: [{ role: "user", content: "안녕!" }],
});
console.log(response.choices[0].message.content);
```
---
## 6. Claude Code CLI
```bash
export ANTHROPIC_BASE_URL=https://cliproxy.gru.farm/api/provider/claude
export ANTHROPIC_API_KEY=Jinie4eva!
claude
```
영구 적용 (`~/.zshrc` 또는 `~/.bashrc`):
```bash
echo 'export ANTHROPIC_BASE_URL=https://cliproxy.gru.farm/api/provider/claude' >> ~/.zshrc
echo 'export ANTHROPIC_API_KEY=Jinie4eva!' >> ~/.zshrc
source ~/.zshrc
```
---
## 7. 환경변수로 관리
`.env` 파일:
```env
ANTHROPIC_BASE_URL=https://cliproxy.gru.farm/api/provider/claude
ANTHROPIC_API_KEY=Jinie4eva!
```
Python에서 `.env` 사용:
```python
from dotenv import load_dotenv
from anthropic import Anthropic
load_dotenv()
# base_url, api_key 자동으로 환경변수에서 읽음
client = Anthropic()
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=1024,
messages=[{"role": "user", "content": "안녕!"}]
)
print(response.content[0].text)
```
---
## 주의사항
- **내부망 접근 시** URL을 `http://192.168.0.17:8317`로 변경
- **OpenAI 호환 모드**는 `/v1/chat/completions`를 사용하지만, Claude 네이티브 기능(extended thinking 등)은 `/api/provider/claude/v1/messages` 사용 권장
- **타임아웃** 설정: 긴 응답의 경우 클라이언트 타임아웃을 600초 이상으로 설정
+232
View File
@@ -0,0 +1,232 @@
# CLIProxyAPI Docker 배포 가이드
NAS(nas.gru.farm)에 Docker로 CLIProxyAPI를 배포하는 방법을 정리합니다.
## 사전 조건
| 항목 | 내용 |
|------|------|
| NAS 접속 | `ssh airkjw@nas.gru.farm -p 22` |
| Docker | `sudo /usr/local/bin/docker` (NOPASSWD) |
| Docker Compose | `sudo /usr/local/bin/docker compose` |
| NAS 내부 IP | 192.168.0.17 |
## 1. 배포 디렉토리 준비
```bash
ssh airkjw@nas.gru.farm
# 배포 디렉토리 생성
mkdir -p ~/docker/cli-proxy-api
cd ~/docker/cli-proxy-api
```
## 2. 필요 파일 구성
NAS에 아래 파일들이 필요합니다:
```
~/docker/cli-proxy-api/
├── docker-compose.yml # 컨테이너 설정
├── config.yaml # 서비스 설정 (API 키, 포트 등)
├── auths/ # OAuth 인증 데이터 (자동 생성)
└── logs/ # 로그 디렉토리 (자동 생성)
```
## 3. docker-compose.yml
로컬 빌드 방식 (소스에서 직접 빌드):
```yaml
services:
cli-proxy-api:
build:
context: .
dockerfile: Dockerfile
container_name: cli-proxy-api
ports:
- "8317:8317" # 메인 API 포트
# 필요시 추가 포트 오픈
# - "8085:8085"
volumes:
- ./config.yaml:/CLIProxyAPI/config.yaml
- ./auths:/root/.cli-proxy-api
- ./logs:/CLIProxyAPI/logs
environment:
- TZ=Asia/Seoul
restart: unless-stopped
```
또는 공식 이미지 사용:
```yaml
services:
cli-proxy-api:
image: eceasy/cli-proxy-api:latest
container_name: cli-proxy-api
ports:
- "8317:8317"
volumes:
- ./config.yaml:/CLIProxyAPI/config.yaml
- ./auths:/root/.cli-proxy-api
- ./logs:/CLIProxyAPI/logs
environment:
- TZ=Asia/Seoul
restart: unless-stopped
```
## 4. config.yaml 설정
`config.example.yaml`을 기반으로 작성합니다.
### 최소 설정 예시
```yaml
# 서버 바인딩
host: ""
port: 8317
# API 키 (클라이언트 인증용, 원하는 값으로 설정)
api-keys:
- "my-secret-api-key-1"
# 디버그 (초기 설정 시 true 권장, 안정화 후 false)
debug: false
# 로그를 파일로 기록
logging-to-file: true
logs-max-total-size-mb: 100
# 재시도 설정
request-retry: 3
```
### Claude API 키 사용 시 추가
```yaml
claude-api-key:
- api-key: "sk-ant-xxxxx"
# base-url: "https://api.anthropic.com" # 기본값이므로 생략 가능
```
### Gemini API 키 사용 시 추가
```yaml
gemini-api-key:
- api-key: "AIzaSy..."
```
### Management UI 활성화 (웹 관리 패널)
```yaml
remote-management:
allow-remote: true
secret-key: "my-management-password"
disable-control-panel: false
```
## 5. 배포 실행
```bash
cd ~/docker/cli-proxy-api
# 공식 이미지 사용 시
sudo /usr/local/bin/docker compose up -d
# 소스 빌드 시 (Gitea에서 소스 가져와서)
git clone http://nas.gru.farm:3001/airkjw/CLIProxyAPI.git src
sudo /usr/local/bin/docker compose -f src/docker-compose.yml up -d --build
```
## 6. 확인
```bash
# 컨테이너 상태 확인
sudo /usr/local/bin/docker ps | grep cli-proxy-api
# 로그 확인
sudo /usr/local/bin/docker logs cli-proxy-api
# API 응답 테스트
curl http://localhost:8317/
curl http://192.168.0.17:8317/
# 모델 목록 확인 (API 키 인증)
curl -H "Authorization: Bearer my-secret-api-key-1" http://localhost:8317/v1/models
```
## 7. 클라이언트 연결
CLIProxyAPI가 실행되면 각 AI CLI 도구에서 프록시 주소로 연결합니다.
### Claude Code에서 사용
```bash
# 환경변수 설정
export ANTHROPIC_BASE_URL=http://192.168.0.17:8317
export ANTHROPIC_API_KEY=my-secret-api-key-1
```
### OpenAI 호환 클라이언트에서 사용
```bash
export OPENAI_BASE_URL=http://192.168.0.17:8317/v1
export OPENAI_API_KEY=my-secret-api-key-1
```
## 8. 관리 & 운영
```bash
# 컨테이너 중지
sudo /usr/local/bin/docker compose down
# 설정 변경 후 재시작
sudo /usr/local/bin/docker compose restart
# 이미지 업데이트 (공식 이미지 사용 시)
sudo /usr/local/bin/docker compose pull
sudo /usr/local/bin/docker compose up -d
# 로그 실시간 모니터링
sudo /usr/local/bin/docker logs -f cli-proxy-api
```
## 포트 목록
| 포트 | 용도 | 필수 여부 |
|------|------|-----------|
| 8317 | 메인 API | 필수 |
| 8085 | 추가 API | 선택 |
| 1455 | 추가 서비스 | 선택 |
| 54545 | 추가 서비스 | 선택 |
| 51121 | 추가 서비스 | 선택 |
| 11451 | 추가 서비스 | 선택 |
> 기본적으로 8317 포트만 열면 됩니다. 나머지는 특정 기능 사용 시 필요합니다.
## 주의사항
- `config.yaml``.gitignore`에 포함되어 있어 Git에 커밋되지 않음 (API 키 보호)
- OAuth 인증(Claude, Gemini 등)은 최초 1회 브라우저 로그인 필요
- `auths/` 디렉토리를 볼륨으로 마운트하면 컨테이너 재생성 시에도 인증 유지
- NAS 외부 접근 시 방화벽/포트포워딩 설정 필요
## 업데이트 이력
| 날짜 | 버전 | 비고 |
|------|------|------|
| 2026-05-04 | v6.10.4 | 69개 커밋 변경 — WebSocket compact 처리 개선, X-Amp-Thread-Id 기반 session affinity, Codex reasoning/이미지 처리 강화, GPT-5.5 모델 추가, OpenAI 호환 provider 비활성화 옵션. 무중단 업데이트, 재인증 불필요 |
| 2026-04-26 | v6.9.38 | Protocol multiplexer + Redis queue 도입, 관리키/Redis AUTH 반복 실패 시 IP 차단 추가. 무중단 업데이트, 재인증 불필요 |
| 2026-04-23 | v6.9.34 | `docker compose pull && docker compose up -d`로 무중단 업데이트. Auth 파일 형식 변경 없어 재인증 불필요 |
| 2026-04-01 | v6.9.7 | 최초 배포 |
### 업데이트 절차
```bash
ssh airkjw@nas.gru.farm
cd /volume2/docker/CLIProxyAPI
sudo /usr/local/bin/docker compose pull
sudo /usr/local/bin/docker compose up -d
```
`auths/` 볼륨이 외부에 마운트되어 있어 컨테이너 교체 시 OAuth 토큰이 유지됩니다.
+17 -5
View File
@@ -1,6 +1,6 @@
# CLI Proxy API
English | [中文](README_CN.md)
English | [中文](README_CN.md) | [日本語](README_JA.md)
A proxy server that provides OpenAI/Gemini/Claude/Codex compatible API interfaces for CLI.
@@ -30,6 +30,14 @@ Get 10% OFF GLM CODING PLANhttps://z.ai/subscribe?ic=8JVLJQFSKB
<td width="180"><a href="https://www.aicodemirror.com/register?invitecode=TJNAIF"><img src="./assets/aicodemirror.png" alt="AICodeMirror" width="150"></a></td>
<td>Thanks to AICodeMirror for sponsoring this project! AICodeMirror provides official high-stability relay services for Claude Code / Codex / Gemini CLI, with enterprise-grade concurrency, fast invoicing, and 24/7 dedicated technical support. Claude Code / Codex / Gemini official channels at 38% / 2% / 9% of original price, with extra discounts on top-ups! AICodeMirror offers special benefits for CLIProxyAPI users: register via <a href="https://www.aicodemirror.com/register?invitecode=TJNAIF">this link</a> to enjoy 20% off your first top-up, and enterprise customers can get up to 25% off!</td>
</tr>
<tr>
<td width="180"><a href="https://shop.bmoplus.com/?utm_source=github"><img src="./assets/bmoplus.png" alt="BmoPlus" width="150"></a></td>
<td>Huge thanks to BmoPlus for sponsoring this project! BmoPlus is a highly reliable AI account provider built strictly for heavy AI users and developers. They offer rock-solid, ready-to-use accounts and official top-up services for ChatGPT Plus / ChatGPT Pro (Full Warranty) / Claude Pro / Super Grok / Gemini Pro. By registering and ordering through <a href="https://shop.bmoplus.com/?utm_source=github">BmoPlus - Premium AI Accounts & Top-ups</a>, users can unlock the mind-blowing rate of <b>10% of the official GPT subscription price (90% OFF)</b>!</td>
</tr>
<tr>
<td width="180"><a href="https://www.lingtrue.com/register"><img src="./assets/lingtrue.png" alt="LingtrueAPI" width="150"></a></td>
<td>Thanks to LingtrueAPI for its sponsorship of this project! LingtrueAPI is a global large - model API intermediary service platform that provides API calling services for various top - notch models such as Claude Code, Codex, and Gemini. It is committed to enabling users to connect to global AI capabilities at low cost and with high stability. LingtrueAPI offers special discounts to users of this software: register using <a href="https://www.lingtrue.com/register">this link</a>, and enter the promo code "LingtrueAPI" when making the first recharge to enjoy a 10% discount.</td>
</tr>
</tbody>
</table>
@@ -74,6 +82,14 @@ CLIProxyAPI includes integrated support for [Amp CLI](https://ampcode.com) and A
- **Model mapping** to route unavailable models to alternatives (e.g., `claude-opus-4.5``claude-sonnet-4`)
- Security-first design with localhost-only management endpoints
When you need the request/response shape of a specific backend family, use the provider-specific paths instead of the merged `/v1/...` endpoints:
- Use `/api/provider/{provider}/v1/messages` for messages-style backends.
- Use `/api/provider/{provider}/v1beta/models/...` for model-scoped generate endpoints.
- Use `/api/provider/{provider}/v1/chat/completions` for chat-completions backends.
These routes help you select the protocol surface, but they do not by themselves guarantee a unique inference executor when the same client-visible model name is reused across multiple backends. Inference routing is still resolved from the request model/alias. For strict backend pinning, use unique aliases, prefixes, or otherwise avoid overlapping client-visible model names.
**→ [Complete Amp CLI Integration Guide](https://help.router-for.me/agent-client/amp-cli.html)**
## SDK Docs
@@ -110,10 +126,6 @@ Browser-based tool to translate SRT subtitles using your Gemini subscription via
CLI wrapper for instant switching between multiple Claude accounts and alternative models (Gemini, Codex, Antigravity) via CLIProxyAPI OAuth - no API keys needed
### [ProxyPal](https://github.com/heyhuynhgiabuu/proxypal)
Native macOS GUI for managing CLIProxyAPI: configure providers, model mappings, and endpoints via OAuth - no API keys needed.
### [Quotio](https://github.com/nguyenphutrong/quotio)
Native macOS menu bar app that unifies Claude, Gemini, OpenAI, Qwen, and Antigravity subscriptions with real-time quota tracking and smart auto-failover for AI coding tools like Claude Code, OpenCode, and Droid - no API keys needed.
+18 -6
View File
@@ -1,6 +1,6 @@
# CLI 代理 API
[English](README.md) | 中文
[English](README.md) | 中文 | [日本語](README_JA.md)
一个为 CLI 提供 OpenAI/Gemini/Claude/Codex 兼容 API 接口的代理服务器。
@@ -30,6 +30,14 @@ GLM CODING PLAN 是专为AI编码打造的订阅套餐,每月最低仅需20元
<td width="180"><a href="https://www.aicodemirror.com/register?invitecode=TJNAIF"><img src="./assets/aicodemirror.png" alt="AICodeMirror" width="150"></a></td>
<td>感谢 AICodeMirror 赞助了本项目!AICodeMirror 提供 Claude Code / Codex / Gemini CLI 官方高稳定中转服务,支持企业级高并发、极速开票、7×24 专属技术支持。 Claude Code / Codex / Gemini 官方渠道低至 3.8 / 0.2 / 0.9 折,充值更有折上折!AICodeMirror 为 CLIProxyAPI 的用户提供了特别福利,通过<a href="https://www.aicodemirror.com/register?invitecode=TJNAIF">此链接</a>注册的用户,可享受首充8折,企业客户最高可享 7.5 折!</td>
</tr>
<tr>
<td width="180"><a href="https://shop.bmoplus.com/?utm_source=github"><img src="./assets/bmoplus.png" alt="BmoPlus" width="150"></a></td>
<td>感谢 BmoPlus 赞助了本项目!BmoPlus 是一家专为AI订阅重度用户打造的可靠 AI 账号代充服务商,提供稳定的 ChatGPT Plus / ChatGPT Pro(全程质保) / Claude Pro / Super Grok / Gemini Pro 的官方代充&成品账号。 通过<a href="https://shop.bmoplus.com/?utm_source=github">BmoPlus AI成品号专卖/代充</a>注册下单的用户,可享GPT <b>官网订阅一折</b> 的震撼价格!</td>
</tr>
<tr>
<td width="180"><a href="https://www.lingtrue.com/register"><img src="./assets/lingtrue.png" alt="LingtrueAPI" width="150"></a></td>
<td>感谢 LingtrueAPI 对本项目的赞助!LingtrueAPI 是一家全球大模型API中转服务平台,提供Claude Code、Codex、Gemini 等多种顶级模型API调用服务,致力于让用户以低成本、高稳定性链接全球AI能力。LingtrueAPI为本软件用户提供了特别优惠:使用<a href="https://www.lingtrue.com/register">此链接</a>注册,并在首次充值时输入 "LingtrueAPI" 优惠码即可享受9折优惠。</td>
</tr>
</tbody>
</table>
@@ -73,6 +81,14 @@ CLIProxyAPI 已内置对 [Amp CLI](https://ampcode.com) 和 Amp IDE 扩展的支
- 智能模型回退与自动路由
- 以安全为先的设计,管理端点仅限 localhost
当你需要某一类后端的请求/响应协议形态时,优先使用 provider-specific 路径,而不是合并后的 `/v1/...` 端点:
- 对于 messages 风格的后端,使用 `/api/provider/{provider}/v1/messages`
- 对于按模型路径暴露生成接口的后端,使用 `/api/provider/{provider}/v1beta/models/...`
- 对于 chat-completions 风格的后端,使用 `/api/provider/{provider}/v1/chat/completions`
这些路径有助于选择协议表面,但当多个后端复用同一个客户端可见模型名时,它们本身并不能保证唯一的推理执行器。实际的推理路由仍然根据请求里的 model/alias 解析。若要严格钉住某个后端,请使用唯一 alias、前缀,或避免让多个后端暴露相同的客户端模型名。
**→ [Amp CLI 完整集成指南](https://help.router-for.me/cn/agent-client/amp-cli.html)**
## SDK 文档
@@ -109,10 +125,6 @@ CLIProxyAPI 已内置对 [Amp CLI](https://ampcode.com) 和 Amp IDE 扩展的支
CLI 封装器,用于通过 CLIProxyAPI OAuth 即时切换多个 Claude 账户和替代模型(Gemini, Codex, Antigravity),无需 API 密钥。
### [ProxyPal](https://github.com/heyhuynhgiabuu/proxypal)
基于 macOS 平台的原生 CLIProxyAPI GUI:配置供应商、模型映射以及OAuth端点,无需 API 密钥。
### [Quotio](https://github.com/nguyenphutrong/quotio)
原生 macOS 菜单栏应用,统一管理 Claude、Gemini、OpenAI、Qwen 和 Antigravity 订阅,提供实时配额追踪和智能自动故障转移,支持 Claude Code、OpenCode 和 Droid 等 AI 编程工具,无需 API 密钥。
@@ -183,7 +195,7 @@ OmniRoute 是一个面向多供应商大语言模型的 AI 网关:它提供兼
## 写给所有中国网友的
QQ 群:188637136
QQ 群:188637136(满)、1081218164
+195
View File
@@ -0,0 +1,195 @@
# CLI Proxy API
[English](README.md) | [中文](README_CN.md) | 日本語
CLI向けのOpenAI/Gemini/Claude/Codex互換APIインターフェースを提供するプロキシサーバーです。
OAuth経由でOpenAI CodexGPTモデル)およびClaude Codeもサポートしています。
ローカルまたはマルチアカウントのCLIアクセスを、OpenAIResponses含む)/Gemini/Claude互換のクライアントやSDKで利用できます。
## スポンサー
[![z.ai](https://assets.router-for.me/english-5-0.jpg)](https://z.ai/subscribe?ic=8JVLJQFSKB)
本プロジェクトはZ.aiにスポンサーされており、GLM CODING PLANの提供を受けています。
GLM CODING PLANはAIコーディング向けに設計されたサブスクリプションサービスで、月額わずか$10から利用可能です。フラッグシップのGLM-4.7および(GLM-5はProユーザーのみ利用可能)モデルを10以上の人気AIコーディングツール(Claude Code、Cline、Roo Codeなど)で利用でき、開発者にトップクラスの高速かつ安定したコーディング体験を提供します。
GLM CODING PLANを10%割引で取得:https://z.ai/subscribe?ic=8JVLJQFSKB
---
<table>
<tbody>
<tr>
<td width="180"><a href="https://www.packyapi.com/register?aff=cliproxyapi"><img src="./assets/packycode.png" alt="PackyCode" width="150"></a></td>
<td>PackyCodeのスポンサーシップに感謝します!PackyCodeは信頼性が高く効率的なAPIリレーサービスプロバイダーで、Claude Code、Codex、Geminiなどのリレーサービスを提供しています。PackyCodeは当ソフトウェアのユーザーに特別割引を提供しています:<a href="https://www.packyapi.com/register?aff=cliproxyapi">こちらのリンク</a>から登録し、チャージ時にプロモーションコード「cliproxyapi」を入力すると10%割引になります。</td>
</tr>
<tr>
<td width="180"><a href="https://www.aicodemirror.com/register?invitecode=TJNAIF"><img src="./assets/aicodemirror.png" alt="AICodeMirror" width="150"></a></td>
<td>AICodeMirrorのスポンサーシップに感謝します!AICodeMirrorはClaude Code / Codex / Gemini CLI向けの公式高安定性リレーサービスを提供しており、エンタープライズグレードの同時接続、迅速な請求書発行、24時間365日の専任技術サポートを備えています。Claude Code / Codex / Geminiの公式チャネルが元の価格の38% / 2% / 9%で利用でき、チャージ時にはさらに割引があります!CLIProxyAPIユーザー向けの特別特典:<a href="https://www.aicodemirror.com/register?invitecode=TJNAIF">こちらのリンク</a>から登録すると、初回チャージが20%割引になり、エンタープライズのお客様は最大25%割引を受けられます!</td>
</tr>
<tr>
<td width="180"><a href="https://shop.bmoplus.com/?utm_source=github"><img src="./assets/bmoplus.png" alt="BmoPlus" width="150"></a></td>
<td>本プロジェクトにご支援いただいた BmoPlus に感謝いたします!BmoPlusは、AIサブスクリプションのヘビーユーザー向けに特化した信頼性の高いAIアカウントサービスプロバイダーであり、安定した ChatGPT Plus / ChatGPT Pro (完全保証) / Claude Pro / Super Grok / Gemini Pro の公式代行チャージおよび即納アカウントを提供しています。こちらの<a href="https://shop.bmoplus.com/?utm_source=github">BmoPlus AIアカウント専門店/代行チャージ</a>経由でご登録・ご注文いただいたユーザー様は、GPTを <b>公式サイト価格の約1割(90% OFF)</b> という驚異的な価格でご利用いただけます!</td>
</tr>
<tr>
<td width="180"><a href="https://www.lingtrue.com/register"><img src="./assets/lingtrue.png" alt="LingtrueAPI" width="150"></a></td>
<td>LingtrueAPIのスポンサーシップに感謝します!LingtrueAPIはグローバルな大規模モデルAPIリレーサービスプラットフォームで、Claude Code、Codex、GeminiなどのトップモデルAPI呼び出しサービスを提供し、ユーザーが低コストかつ高い安定性で世界中のAI能力に接続できるよう支援しています。LingtrueAPIは本ソフトウェアのユーザーに特別割引を提供しています:<a href="https://www.lingtrue.com/register">こちらのリンク</a>から登録し、初回チャージ時にプロモーションコード「LingtrueAPI」を入力すると10%割引になります。</td>
</tr>
</tbody>
</table>
## 概要
- CLIモデル向けのOpenAI/Gemini/Claude互換APIエンドポイント
- OAuthログインによるOpenAI Codexサポート(GPTモデル)
- OAuthログインによるClaude Codeサポート
- OAuthログインによるQwen Codeサポート
- OAuthログインによるiFlowサポート
- プロバイダールーティングによるAmp CLIおよびIDE拡張機能のサポート
- ストリーミングおよび非ストリーミングレスポンス
- 関数呼び出し/ツールのサポート
- マルチモーダル入力サポート(テキストと画像)
- ラウンドロビン負荷分散による複数アカウント対応(Gemini、OpenAI、Claude、QwenおよびiFlow
- シンプルなCLI認証フロー(Gemini、OpenAI、Claude、QwenおよびiFlow
- Generative Language APIキーのサポート
- AI Studioビルドのマルチアカウント負荷分散
- Gemini CLIのマルチアカウント負荷分散
- Claude Codeのマルチアカウント負荷分散
- Qwen Codeのマルチアカウント負荷分散
- iFlowのマルチアカウント負荷分散
- OpenAI Codexのマルチアカウント負荷分散
- 設定によるOpenAI互換アップストリームプロバイダー(例:OpenRouter)
- プロキシ埋め込み用の再利用可能なGo SDK(`docs/sdk-usage.md`を参照)
## はじめに
CLIProxyAPIガイド:[https://help.router-for.me/](https://help.router-for.me/)
## 管理API
[MANAGEMENT_API.md](https://help.router-for.me/management/api)を参照
## Amp CLIサポート
CLIProxyAPIは[Amp CLI](https://ampcode.com)およびAmp IDE拡張機能の統合サポートを含んでおり、Google/ChatGPT/ClaudeのOAuthサブスクリプションをAmpのコーディングツールで使用できます:
- Ampの APIパターン用のプロバイダールートエイリアス(`/api/provider/{provider}/v1...`
- OAuth認証およびアカウント機能用の管理プロキシ
- 自動ルーティングによるスマートモデルフォールバック
- 利用できないモデルを代替モデルにルーティングする**モデルマッピング**(例:`claude-opus-4.5``claude-sonnet-4`
- localhostのみの管理エンドポイントによるセキュリティファーストの設計
特定のバックエンド系統のリクエスト/レスポンス形状が必要な場合は、統合された `/v1/...` エンドポイントよりも provider-specific のパスを優先してください。
- messages 系のバックエンドには `/api/provider/{provider}/v1/messages`
- モデル単位の generate 系エンドポイントには `/api/provider/{provider}/v1beta/models/...`
- chat-completions 系のバックエンドには `/api/provider/{provider}/v1/chat/completions`
これらのパスはプロトコル面の選択には役立ちますが、同じクライアント向けモデル名が複数バックエンドで再利用されている場合、それだけで推論実行系が一意に固定されるわけではありません。実際の推論ルーティングは、引き続きリクエスト内の model/alias 解決に従います。厳密にバックエンドを固定したい場合は、一意な alias や prefix を使うか、クライアント向けモデル名の重複自体を避けてください。
**→ [Amp CLI統合ガイドの完全版](https://help.router-for.me/agent-client/amp-cli.html)**
## SDKドキュメント
- 使い方:[docs/sdk-usage.md](docs/sdk-usage.md)
- 上級(エグゼキューターとトランスレーター):[docs/sdk-advanced.md](docs/sdk-advanced.md)
- アクセス:[docs/sdk-access.md](docs/sdk-access.md)
- ウォッチャー:[docs/sdk-watcher.md](docs/sdk-watcher.md)
- カスタムプロバイダーの例:`examples/custom-provider`
## コントリビューション
コントリビューションを歓迎します!お気軽にPull Requestを送ってください。
1. リポジトリをフォーク
2. フィーチャーブランチを作成(`git checkout -b feature/amazing-feature`
3. 変更をコミット(`git commit -m 'Add some amazing feature'`
4. ブランチにプッシュ(`git push origin feature/amazing-feature`
5. Pull Requestを作成
## 関連プロジェクト
CLIProxyAPIをベースにした以下のプロジェクトがあります:
### [vibeproxy](https://github.com/automazeio/vibeproxy)
macOSネイティブのメニューバーアプリで、Claude CodeとChatGPTのサブスクリプションをAIコーディングツールで使用可能 - APIキー不要
### [Subtitle Translator](https://github.com/VjayC/SRT-Subtitle-Translator-Validator)
CLIProxyAPI経由でGeminiサブスクリプションを使用してSRT字幕を翻訳するブラウザベースのツール。自動検証/エラー修正機能付き - APIキー不要
### [CCS (Claude Code Switch)](https://github.com/kaitranntt/ccs)
CLIProxyAPI OAuthを使用して複数のClaudeアカウントや代替モデル(Gemini、Codex、Antigravity)を即座に切り替えるCLIラッパー - APIキー不要
### [Quotio](https://github.com/nguyenphutrong/quotio)
Claude、Gemini、OpenAI、Qwen、Antigravityのサブスクリプションを統合し、リアルタイムのクォータ追跡とスマート自動フェイルオーバーを備えたmacOSネイティブのメニューバーアプリ。Claude Code、OpenCode、Droidなどのコーディングツール向け - APIキー不要
### [CodMate](https://github.com/loocor/CodMate)
CLI AIセッション(Codex、Claude Code、Gemini CLI)を管理するmacOS SwiftUIネイティブアプリ。統合プロバイダー管理、Gitレビュー、プロジェクト整理、グローバル検索、ターミナル統合機能を搭載。CLIProxyAPIと統合し、Codex、Claude、Gemini、Antigravity、Qwen CodeのOAuth認証を提供。単一のプロキシエンドポイントを通じた組み込みおよびサードパーティプロバイダーの再ルーティングに対応 - OAuthプロバイダーではAPIキー不要
### [ProxyPilot](https://github.com/Finesssee/ProxyPilot)
TUI、システムトレイ、マルチプロバイダーOAuthを備えたWindows向けCLIProxyAPIフォーク - AIコーディングツール用、APIキー不要
### [Claude Proxy VSCode](https://github.com/uzhao/claude-proxy-vscode)
Claude Codeモデルを素早く切り替えるVSCode拡張機能。バックエンドとしてCLIProxyAPIを統合し、バックグラウンドでの自動ライフサイクル管理を搭載
### [ZeroLimit](https://github.com/0xtbug/zero-limit)
CLIProxyAPIを使用してAIコーディングアシスタントのクォータを監視するTauri + React製のWindowsデスクトップアプリ。Gemini、Claude、OpenAI Codex、Antigravityアカウントの使用量をリアルタイムダッシュボード、システムトレイ統合、ワンクリックプロキシコントロールで追跡 - APIキー不要
### [CPA-XXX Panel](https://github.com/ferretgeek/CPA-X)
CLIProxyAPI向けの軽量Web管理パネル。ヘルスチェック、リソース監視、リアルタイムログ、自動更新、リクエスト統計、料金表示機能を搭載。ワンクリックインストールとsystemdサービスに対応
### [CLIProxyAPI Tray](https://github.com/kitephp/CLIProxyAPI_Tray)
PowerShellスクリプトで実装されたWindowsトレイアプリケーション。サードパーティライブラリに依存せず、ショートカットの自動作成、サイレント実行、パスワード管理、チャネル切り替え(Main / Plus)、自動ダウンロードおよび自動更新に対応
### [霖君](https://github.com/wangdabaoqq/LinJun)
霖君はAIプログラミングアシスタントを管理するクロスプラットフォームデスクトップアプリケーションで、macOS、Windows、Linuxシステムに対応。Claude Code、Gemini CLI、OpenAI Codex、Qwen Codeなどのコーディングツールを統合管理し、ローカルプロキシによるマルチアカウントクォータ追跡とワンクリック設定が可能
### [CLIProxyAPI Dashboard](https://github.com/itsmylife44/cliproxyapi-dashboard)
Next.js、React、PostgreSQLで構築されたCLIProxyAPI用のモダンなWebベース管理ダッシュボード。リアルタイムログストリーミング、構造化された設定編集、APIキー管理、Claude/Gemini/Codex向けOAuthプロバイダー統合、使用量分析、コンテナ管理、コンパニオンプラグインによるOpenCodeとの設定同期機能を搭載 - 手動でのYAML編集は不要
### [All API Hub](https://github.com/qixing-jk/all-api-hub)
New API互換リレーサイトアカウントをワンストップで管理するブラウザ拡張機能。残高と使用量のダッシュボード、自動チェックイン、一般的なアプリへのワンクリックキーエクスポート、ページ内API可用性テスト、チャネル/モデルの同期とリダイレクト機能を搭載。Management APIを通じてCLIProxyAPIと統合し、ワンクリックでプロバイダーのインポートと設定同期が可能
### [Shadow AI](https://github.com/HEUDavid/shadow-ai)
Shadow AIは制限された環境向けに特別に設計されたAIアシスタントツールです。ウィンドウや痕跡のないステルス動作モードを提供し、LAN(ローカルエリアネットワーク)を介したクロスデバイスAI質疑応答のインタラクションと制御を可能にします。本質的には「画面/音声キャプチャ + AI推論 + 低摩擦デリバリー」の自動化コラボレーションレイヤーであり、制御されたデバイスや制限された環境でアプリケーション横断的にAIアシスタントを没入的に使用できるようユーザーを支援します。
> [!NOTE]
> CLIProxyAPIをベースにプロジェクトを開発した場合は、PRを送ってこのリストに追加してください。
## その他の選択肢
以下のプロジェクトはCLIProxyAPIの移植版またはそれに触発されたものです:
### [9Router](https://github.com/decolua/9router)
CLIProxyAPIに触発されたNext.js実装。インストールと使用が簡単で、フォーマット変換(OpenAI/Claude/Gemini/Ollama)、自動フォールバック付きコンボシステム、指数バックオフ付きマルチアカウント管理、Next.js Webダッシュボード、CLIツール(Cursor、Claude Code、Cline、RooCode)のサポートをゼロから構築 - APIキー不要
### [OmniRoute](https://github.com/diegosouzapw/OmniRoute)
コーディングを止めない。無料および低コストのAIモデルへのスマートルーティングと自動フォールバック。
OmniRouteはマルチプロバイダーLLM向けのAIゲートウェイです:スマートルーティング、負荷分散、リトライ、フォールバックを備えたOpenAI互換エンドポイント。ポリシー、レート制限、キャッシュ、可観測性を追加して、信頼性が高くコストを意識した推論を実現します。
> [!NOTE]
> CLIProxyAPIの移植版またはそれに触発されたプロジェクトを開発した場合は、PRを送ってこのリストに追加してください。
## ライセンス
本プロジェクトはMITライセンスの下でライセンスされています - 詳細は[LICENSE](LICENSE)ファイルを参照してください。
+104
View File
@@ -0,0 +1,104 @@
# CLIProxyAPI 역방향 프록시 & HTTPS 설정 가이드
외부에서 `https://cliproxy.gru.farm`으로 CLIProxyAPI에 접근하기 위한 설정입니다.
## 1단계: DNS 레코드 추가
hostcocoa.com DNS 관리에서 A 레코드를 추가합니다.
| 타입 | 호스트 | 값 |
|------|--------|-----|
| A | cliproxy | 125.188.185.74 |
> 기존 `nas.gru.farm`, `haesol.gru.farm` 등과 같은 IP입니다.
## 2단계: Synology DSM 역방향 프록시 설정
1. DSM 웹 UI 접속 (보통 `https://nas.gru.farm:5001`)
2. **제어판****로그인 포털****고급** 탭 → **역방향 프록시** 클릭
3. **생성** 버튼 클릭
4. 아래와 같이 입력:
### 일반 설정
| 항목 | 값 |
|------|-----|
| 설명 | `CLIProxyAPI` |
| **소스 (프론트엔드)** | |
| 프로토콜 | `HTTPS` |
| 호스트 이름 | `cliproxy.gru.farm` |
| 포트 | `443` |
| HSTS | 비활성화 |
| **대상 (백엔드)** | |
| 프로토콜 | `HTTP` |
| 호스트 이름 | `localhost` |
| 포트 | `8317` |
### 사용자 지정 헤더 (선택)
필요 시 WebSocket 지원을 위해 사용자 지정 헤더 추가:
- `Upgrade``$http_upgrade`
- `Connection``$connection_upgrade`
### 타임아웃 설정
AI 요청은 응답이 오래 걸릴 수 있으므로 타임아웃을 늘려주세요:
- 연결 타임아웃: `600`
- 전송 타임아웃: `600`
- 수신 타임아웃: `600`
5. **저장** 클릭
## 3단계: SSL 인증서 설정
Synology DSM에서 `cliproxy.gru.farm` 용 SSL 인증서를 설정합니다.
### Let's Encrypt 인증서 발급 (권장)
1. **제어판****보안****인증서**
2. **추가****새 인증서 추가****Let's Encrypt에서 인증서 가져오기**
3. 도메인: `cliproxy.gru.farm`
4. 이메일: 본인 이메일
5. 발급 완료 후, **설정** 버튼 클릭
6. `cliproxy.gru.farm` 역방향 프록시 항목에 방금 발급한 인증서 선택
### 기존 와일드카드 인증서가 있는 경우
`*.gru.farm` 와일드카드 인증서가 있다면 별도 발급 없이 해당 인증서를 선택하면 됩니다.
## 4단계: 공유기 포트 포워딩
공유기에서 443 포트가 NAS(192.168.0.17)로 포워딩되어 있는지 확인합니다.
> 기존 `haesol.gru.farm` 등이 HTTPS로 동작 중이라면 이미 설정되어 있을 가능성이 높습니다.
| 외부 포트 | 내부 IP | 내부 포트 | 프로토콜 |
|-----------|---------|-----------|----------|
| 443 | 192.168.0.17 | 443 | TCP |
## 5단계: 확인
```bash
# DNS 전파 확인
dig +short cliproxy.gru.farm
# 125.188.185.74 가 나오면 성공
# HTTPS 접속 테스트
curl https://cliproxy.gru.farm/
# {"endpoints":[...],"message":"CLI Proxy API Server"}
# 모델 목록 확인
curl -H "Authorization: Bearer Jinie4eva!" https://cliproxy.gru.farm/v1/models
```
## 클라이언트 연결 (외부)
```bash
# Claude Code
export ANTHROPIC_BASE_URL=https://cliproxy.gru.farm
export ANTHROPIC_API_KEY=Jinie4eva!
# OpenAI 호환
export OPENAI_BASE_URL=https://cliproxy.gru.farm/v1
export OPENAI_API_KEY=Jinie4eva!
```
Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 129 KiB

+19 -2
View File
@@ -25,6 +25,10 @@ remote-management:
# Disable the bundled management control panel asset download and HTTP route when true.
disable-control-panel: false
# Disable automatic periodic background updates of the management panel from GitHub (default: false).
# When enabled, the panel is only downloaded on first access if missing, and never auto-updated afterward.
# disable-auto-update-panel: false
# GitHub repository for the management control panel. Accepts a repository URL or releases API URL.
panel-github-repository: "https://github.com/router-for-me/Cli-Proxy-API-Management-Center"
@@ -170,12 +174,19 @@ nonstream-keepalive-interval: 0
# cache-user-id: true # optional: default is false; set true to reuse cached user_id per API key instead of generating a random one each request
# Default headers for Claude API requests. Update when Claude Code releases new versions.
# These are used as fallbacks when the client does not send its own headers.
# In legacy mode, user-agent/package-version/runtime-version/timeout are used as fallbacks
# when the client omits them, while OS/arch remain runtime-derived. When
# stabilize-device-profile is enabled, OS/arch stay pinned to the baseline values below,
# while user-agent/package-version/runtime-version seed a software fingerprint that can
# still upgrade to newer official Claude client versions.
# claude-header-defaults:
# user-agent: "claude-cli/2.1.44 (external, sdk-cli)"
# package-version: "0.74.0"
# runtime-version: "v24.3.0"
# os: "MacOS"
# arch: "arm64"
# timeout: "600"
# stabilize-device-profile: false # optional, default false; set true to enable per-auth/API-key fingerprint pinning
# Default headers for Codex OAuth model requests.
# These are used only for file-backed/OAuth Codex requests when the client
@@ -199,7 +210,9 @@ nonstream-keepalive-interval: 0
# - api-key: "sk-or-v1-...b781" # without proxy-url
# models: # The models supported by the provider.
# - name: "moonshotai/kimi-k2:free" # The actual model name.
# alias: "kimi-k2" # The alias used in the API.
# alias: "kimi-k2" # The alias used in the API.
# thinking: # optional: omit to default to levels ["low","medium","high"]
# levels: ["low", "medium", "high"]
# # You may repeat the same alias to build an internal model pool.
# # The client still sees only one alias in the model list.
# # Requests to that alias will round-robin across the upstream names below,
@@ -268,6 +281,10 @@ nonstream-keepalive-interval: 0
# These aliases rename model IDs for both model listing and request routing.
# Supported channels: gemini-cli, vertex, aistudio, antigravity, claude, codex, qwen, iflow, kimi.
# NOTE: Aliases do not apply to gemini-api-key, codex-api-key, claude-api-key, openai-compatibility, vertex-api-key, or ampcode.
# NOTE: Because aliases affect the merged /v1 model list and merged request routing, overlapping
# client-visible names can become ambiguous across providers. /api/provider/{provider}/... helps
# you select the protocol surface, but inference backend selection can still follow the resolved
# model/alias. For strict backend pinning, use unique aliases/prefixes or avoid overlapping names.
# You can repeat the same name with different aliases to expose multiple client model names.
# oauth-model-alias:
# gemini-cli:
+295 -61
View File
@@ -9,6 +9,7 @@ import (
"errors"
"fmt"
"io"
"mime/multipart"
"net"
"net/http"
"os"
@@ -57,8 +58,10 @@ type callbackForwarder struct {
}
var (
callbackForwardersMu sync.Mutex
callbackForwarders = make(map[int]*callbackForwarder)
callbackForwardersMu sync.Mutex
callbackForwarders = make(map[int]*callbackForwarder)
errAuthFileMustBeJSON = errors.New("auth file must be .json")
errAuthFileNotFound = errors.New("auth file not found")
)
func extractLastRefreshTimestamp(meta map[string]any) (time.Time, bool) {
@@ -538,10 +541,23 @@ func isRuntimeOnlyAuth(auth *coreauth.Auth) bool {
return strings.EqualFold(strings.TrimSpace(auth.Attributes["runtime_only"]), "true")
}
func isUnsafeAuthFileName(name string) bool {
if strings.TrimSpace(name) == "" {
return true
}
if strings.ContainsAny(name, "/\\") {
return true
}
if filepath.VolumeName(name) != "" {
return true
}
return false
}
// Download single auth file by name
func (h *Handler) DownloadAuthFile(c *gin.Context) {
name := c.Query("name")
if name == "" || strings.Contains(name, string(os.PathSeparator)) {
name := strings.TrimSpace(c.Query("name"))
if isUnsafeAuthFileName(name) {
c.JSON(400, gin.H{"error": "invalid name"})
return
}
@@ -570,36 +586,61 @@ func (h *Handler) UploadAuthFile(c *gin.Context) {
return
}
ctx := c.Request.Context()
if file, err := c.FormFile("file"); err == nil && file != nil {
name := filepath.Base(file.Filename)
if !strings.HasSuffix(strings.ToLower(name), ".json") {
c.JSON(400, gin.H{"error": "file must be .json"})
return
}
dst := filepath.Join(h.cfg.AuthDir, name)
if !filepath.IsAbs(dst) {
if abs, errAbs := filepath.Abs(dst); errAbs == nil {
dst = abs
}
}
if errSave := c.SaveUploadedFile(file, dst); errSave != nil {
c.JSON(500, gin.H{"error": fmt.Sprintf("failed to save file: %v", errSave)})
return
}
data, errRead := os.ReadFile(dst)
if errRead != nil {
c.JSON(500, gin.H{"error": fmt.Sprintf("failed to read saved file: %v", errRead)})
return
}
if errReg := h.registerAuthFromFile(ctx, dst, data); errReg != nil {
c.JSON(500, gin.H{"error": errReg.Error()})
return
}
c.JSON(200, gin.H{"status": "ok"})
fileHeaders, errMultipart := h.multipartAuthFileHeaders(c)
if errMultipart != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": fmt.Sprintf("invalid multipart form: %v", errMultipart)})
return
}
name := c.Query("name")
if name == "" || strings.Contains(name, string(os.PathSeparator)) {
if len(fileHeaders) == 1 {
if _, errUpload := h.storeUploadedAuthFile(ctx, fileHeaders[0]); errUpload != nil {
if errors.Is(errUpload, errAuthFileMustBeJSON) {
c.JSON(http.StatusBadRequest, gin.H{"error": "file must be .json"})
return
}
c.JSON(http.StatusInternalServerError, gin.H{"error": errUpload.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"status": "ok"})
return
}
if len(fileHeaders) > 1 {
uploaded := make([]string, 0, len(fileHeaders))
failed := make([]gin.H, 0)
for _, file := range fileHeaders {
name, errUpload := h.storeUploadedAuthFile(ctx, file)
if errUpload != nil {
failureName := ""
if file != nil {
failureName = filepath.Base(file.Filename)
}
msg := errUpload.Error()
if errors.Is(errUpload, errAuthFileMustBeJSON) {
msg = "file must be .json"
}
failed = append(failed, gin.H{"name": failureName, "error": msg})
continue
}
uploaded = append(uploaded, name)
}
if len(failed) > 0 {
c.JSON(http.StatusMultiStatus, gin.H{
"status": "partial",
"uploaded": len(uploaded),
"files": uploaded,
"failed": failed,
})
return
}
c.JSON(http.StatusOK, gin.H{"status": "ok", "uploaded": len(uploaded), "files": uploaded})
return
}
if c.ContentType() == "multipart/form-data" {
c.JSON(http.StatusBadRequest, gin.H{"error": "no files uploaded"})
return
}
name := strings.TrimSpace(c.Query("name"))
if isUnsafeAuthFileName(name) {
c.JSON(400, gin.H{"error": "invalid name"})
return
}
@@ -612,17 +653,7 @@ func (h *Handler) UploadAuthFile(c *gin.Context) {
c.JSON(400, gin.H{"error": "failed to read body"})
return
}
dst := filepath.Join(h.cfg.AuthDir, filepath.Base(name))
if !filepath.IsAbs(dst) {
if abs, errAbs := filepath.Abs(dst); errAbs == nil {
dst = abs
}
}
if errWrite := os.WriteFile(dst, data, 0o600); errWrite != nil {
c.JSON(500, gin.H{"error": fmt.Sprintf("failed to write file: %v", errWrite)})
return
}
if err = h.registerAuthFromFile(ctx, dst, data); err != nil {
if err = h.writeAuthFile(ctx, filepath.Base(name), data); err != nil {
c.JSON(500, gin.H{"error": err.Error()})
return
}
@@ -669,11 +700,182 @@ func (h *Handler) DeleteAuthFile(c *gin.Context) {
c.JSON(200, gin.H{"status": "ok", "deleted": deleted})
return
}
name := c.Query("name")
if name == "" || strings.Contains(name, string(os.PathSeparator)) {
names, errNames := requestedAuthFileNamesForDelete(c)
if errNames != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": errNames.Error()})
return
}
if len(names) == 0 {
c.JSON(400, gin.H{"error": "invalid name"})
return
}
if len(names) == 1 {
if _, status, errDelete := h.deleteAuthFileByName(ctx, names[0]); errDelete != nil {
c.JSON(status, gin.H{"error": errDelete.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"status": "ok"})
return
}
deletedFiles := make([]string, 0, len(names))
failed := make([]gin.H, 0)
for _, name := range names {
deletedName, _, errDelete := h.deleteAuthFileByName(ctx, name)
if errDelete != nil {
failed = append(failed, gin.H{"name": name, "error": errDelete.Error()})
continue
}
deletedFiles = append(deletedFiles, deletedName)
}
if len(failed) > 0 {
c.JSON(http.StatusMultiStatus, gin.H{
"status": "partial",
"deleted": len(deletedFiles),
"files": deletedFiles,
"failed": failed,
})
return
}
c.JSON(http.StatusOK, gin.H{"status": "ok", "deleted": len(deletedFiles), "files": deletedFiles})
}
func (h *Handler) multipartAuthFileHeaders(c *gin.Context) ([]*multipart.FileHeader, error) {
if h == nil || c == nil || c.ContentType() != "multipart/form-data" {
return nil, nil
}
form, err := c.MultipartForm()
if err != nil {
return nil, err
}
if form == nil || len(form.File) == 0 {
return nil, nil
}
keys := make([]string, 0, len(form.File))
for key := range form.File {
keys = append(keys, key)
}
sort.Strings(keys)
headers := make([]*multipart.FileHeader, 0)
for _, key := range keys {
headers = append(headers, form.File[key]...)
}
return headers, nil
}
func (h *Handler) storeUploadedAuthFile(ctx context.Context, file *multipart.FileHeader) (string, error) {
if file == nil {
return "", fmt.Errorf("no file uploaded")
}
name := filepath.Base(strings.TrimSpace(file.Filename))
if !strings.HasSuffix(strings.ToLower(name), ".json") {
return "", errAuthFileMustBeJSON
}
src, err := file.Open()
if err != nil {
return "", fmt.Errorf("failed to open uploaded file: %w", err)
}
defer src.Close()
data, err := io.ReadAll(src)
if err != nil {
return "", fmt.Errorf("failed to read uploaded file: %w", err)
}
if err := h.writeAuthFile(ctx, name, data); err != nil {
return "", err
}
return name, nil
}
func (h *Handler) writeAuthFile(ctx context.Context, name string, data []byte) error {
dst := filepath.Join(h.cfg.AuthDir, filepath.Base(name))
if !filepath.IsAbs(dst) {
if abs, errAbs := filepath.Abs(dst); errAbs == nil {
dst = abs
}
}
auth, err := h.buildAuthFromFileData(dst, data)
if err != nil {
return err
}
if errWrite := os.WriteFile(dst, data, 0o600); errWrite != nil {
return fmt.Errorf("failed to write file: %w", errWrite)
}
if err := h.upsertAuthRecord(ctx, auth); err != nil {
return err
}
return nil
}
func requestedAuthFileNamesForDelete(c *gin.Context) ([]string, error) {
if c == nil {
return nil, nil
}
names := uniqueAuthFileNames(c.QueryArray("name"))
if len(names) > 0 {
return names, nil
}
body, err := io.ReadAll(c.Request.Body)
if err != nil {
return nil, fmt.Errorf("failed to read body")
}
body = bytes.TrimSpace(body)
if len(body) == 0 {
return nil, nil
}
var objectBody struct {
Name string `json:"name"`
Names []string `json:"names"`
}
if body[0] == '[' {
var arrayBody []string
if err := json.Unmarshal(body, &arrayBody); err != nil {
return nil, fmt.Errorf("invalid request body")
}
return uniqueAuthFileNames(arrayBody), nil
}
if err := json.Unmarshal(body, &objectBody); err != nil {
return nil, fmt.Errorf("invalid request body")
}
out := make([]string, 0, len(objectBody.Names)+1)
if strings.TrimSpace(objectBody.Name) != "" {
out = append(out, objectBody.Name)
}
out = append(out, objectBody.Names...)
return uniqueAuthFileNames(out), nil
}
func uniqueAuthFileNames(names []string) []string {
if len(names) == 0 {
return nil
}
seen := make(map[string]struct{}, len(names))
out := make([]string, 0, len(names))
for _, name := range names {
name = strings.TrimSpace(name)
if name == "" {
continue
}
if _, ok := seen[name]; ok {
continue
}
seen[name] = struct{}{}
out = append(out, name)
}
return out
}
func (h *Handler) deleteAuthFileByName(ctx context.Context, name string) (string, int, error) {
name = strings.TrimSpace(name)
if isUnsafeAuthFileName(name) {
return "", http.StatusBadRequest, fmt.Errorf("invalid name")
}
targetPath := filepath.Join(h.cfg.AuthDir, filepath.Base(name))
targetID := ""
@@ -690,22 +892,19 @@ func (h *Handler) DeleteAuthFile(c *gin.Context) {
}
if errRemove := os.Remove(targetPath); errRemove != nil {
if os.IsNotExist(errRemove) {
c.JSON(404, gin.H{"error": "file not found"})
} else {
c.JSON(500, gin.H{"error": fmt.Sprintf("failed to remove file: %v", errRemove)})
return filepath.Base(name), http.StatusNotFound, errAuthFileNotFound
}
return
return filepath.Base(name), http.StatusInternalServerError, fmt.Errorf("failed to remove file: %w", errRemove)
}
if errDeleteRecord := h.deleteTokenRecord(ctx, targetPath); errDeleteRecord != nil {
c.JSON(500, gin.H{"error": errDeleteRecord.Error()})
return
return filepath.Base(name), http.StatusInternalServerError, errDeleteRecord
}
if targetID != "" {
h.disableAuth(ctx, targetID)
} else {
h.disableAuth(ctx, targetPath)
}
c.JSON(200, gin.H{"status": "ok"})
return filepath.Base(name), http.StatusOK, nil
}
func (h *Handler) findAuthForDelete(name string) *coreauth.Auth {
@@ -739,10 +938,25 @@ func (h *Handler) authIDForPath(path string) string {
if path == "" {
return ""
}
path = filepath.Clean(path)
if !filepath.IsAbs(path) {
if abs, errAbs := filepath.Abs(path); errAbs == nil {
path = abs
}
}
id := path
if h != nil && h.cfg != nil {
authDir := strings.TrimSpace(h.cfg.AuthDir)
if resolvedAuthDir, errResolve := util.ResolveAuthDir(authDir); errResolve == nil && resolvedAuthDir != "" {
authDir = resolvedAuthDir
}
if authDir != "" {
authDir = filepath.Clean(authDir)
if !filepath.IsAbs(authDir) {
if abs, errAbs := filepath.Abs(authDir); errAbs == nil {
authDir = abs
}
}
if rel, errRel := filepath.Rel(authDir, path); errRel == nil && rel != "" {
id = rel
}
@@ -759,19 +973,27 @@ func (h *Handler) registerAuthFromFile(ctx context.Context, path string, data []
if h.authManager == nil {
return nil
}
auth, err := h.buildAuthFromFileData(path, data)
if err != nil {
return err
}
return h.upsertAuthRecord(ctx, auth)
}
func (h *Handler) buildAuthFromFileData(path string, data []byte) (*coreauth.Auth, error) {
if path == "" {
return fmt.Errorf("auth path is empty")
return nil, fmt.Errorf("auth path is empty")
}
if data == nil {
var err error
data, err = os.ReadFile(path)
if err != nil {
return fmt.Errorf("failed to read auth file: %w", err)
return nil, fmt.Errorf("failed to read auth file: %w", err)
}
}
metadata := make(map[string]any)
if err := json.Unmarshal(data, &metadata); err != nil {
return fmt.Errorf("invalid auth file: %w", err)
return nil, fmt.Errorf("invalid auth file: %w", err)
}
provider, _ := metadata["type"].(string)
if provider == "" {
@@ -805,13 +1027,25 @@ func (h *Handler) registerAuthFromFile(ctx context.Context, path string, data []
if hasLastRefresh {
auth.LastRefreshedAt = lastRefresh
}
if existing, ok := h.authManager.GetByID(authID); ok {
auth.CreatedAt = existing.CreatedAt
if !hasLastRefresh {
auth.LastRefreshedAt = existing.LastRefreshedAt
if h != nil && h.authManager != nil {
if existing, ok := h.authManager.GetByID(authID); ok {
auth.CreatedAt = existing.CreatedAt
if !hasLastRefresh {
auth.LastRefreshedAt = existing.LastRefreshedAt
}
auth.NextRefreshAfter = existing.NextRefreshAfter
auth.Runtime = existing.Runtime
}
auth.NextRefreshAfter = existing.NextRefreshAfter
auth.Runtime = existing.Runtime
}
return auth, nil
}
func (h *Handler) upsertAuthRecord(ctx context.Context, auth *coreauth.Auth) error {
if h == nil || h.authManager == nil || auth == nil {
return nil
}
if existing, ok := h.authManager.GetByID(auth.ID); ok {
auth.CreatedAt = existing.CreatedAt
_, err := h.authManager.Update(ctx, auth)
return err
}
@@ -0,0 +1,197 @@
package management
import (
"bytes"
"encoding/json"
"mime/multipart"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"testing"
"github.com/gin-gonic/gin"
"github.com/router-for-me/CLIProxyAPI/v6/internal/config"
coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
)
func TestUploadAuthFile_BatchMultipart(t *testing.T) {
t.Setenv("MANAGEMENT_PASSWORD", "")
gin.SetMode(gin.TestMode)
authDir := t.TempDir()
manager := coreauth.NewManager(nil, nil, nil)
h := NewHandlerWithoutConfigFilePath(&config.Config{AuthDir: authDir}, manager)
files := []struct {
name string
content string
}{
{name: "alpha.json", content: `{"type":"codex","email":"alpha@example.com"}`},
{name: "beta.json", content: `{"type":"claude","email":"beta@example.com"}`},
}
var body bytes.Buffer
writer := multipart.NewWriter(&body)
for _, file := range files {
part, err := writer.CreateFormFile("file", file.name)
if err != nil {
t.Fatalf("failed to create multipart file: %v", err)
}
if _, err = part.Write([]byte(file.content)); err != nil {
t.Fatalf("failed to write multipart content: %v", err)
}
}
if err := writer.Close(); err != nil {
t.Fatalf("failed to close multipart writer: %v", err)
}
rec := httptest.NewRecorder()
ctx, _ := gin.CreateTestContext(rec)
req := httptest.NewRequest(http.MethodPost, "/v0/management/auth-files", &body)
req.Header.Set("Content-Type", writer.FormDataContentType())
ctx.Request = req
h.UploadAuthFile(ctx)
if rec.Code != http.StatusOK {
t.Fatalf("expected upload status %d, got %d with body %s", http.StatusOK, rec.Code, rec.Body.String())
}
var payload map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &payload); err != nil {
t.Fatalf("failed to decode response: %v", err)
}
if got, ok := payload["uploaded"].(float64); !ok || int(got) != len(files) {
t.Fatalf("expected uploaded=%d, got %#v", len(files), payload["uploaded"])
}
for _, file := range files {
fullPath := filepath.Join(authDir, file.name)
data, err := os.ReadFile(fullPath)
if err != nil {
t.Fatalf("expected uploaded file %s to exist: %v", file.name, err)
}
if string(data) != file.content {
t.Fatalf("expected file %s content %q, got %q", file.name, file.content, string(data))
}
}
auths := manager.List()
if len(auths) != len(files) {
t.Fatalf("expected %d auth entries, got %d", len(files), len(auths))
}
}
func TestUploadAuthFile_BatchMultipart_InvalidJSONDoesNotOverwriteExistingFile(t *testing.T) {
t.Setenv("MANAGEMENT_PASSWORD", "")
gin.SetMode(gin.TestMode)
authDir := t.TempDir()
manager := coreauth.NewManager(nil, nil, nil)
h := NewHandlerWithoutConfigFilePath(&config.Config{AuthDir: authDir}, manager)
existingName := "alpha.json"
existingContent := `{"type":"codex","email":"alpha@example.com"}`
if err := os.WriteFile(filepath.Join(authDir, existingName), []byte(existingContent), 0o600); err != nil {
t.Fatalf("failed to seed existing auth file: %v", err)
}
files := []struct {
name string
content string
}{
{name: existingName, content: `{"type":"codex"`},
{name: "beta.json", content: `{"type":"claude","email":"beta@example.com"}`},
}
var body bytes.Buffer
writer := multipart.NewWriter(&body)
for _, file := range files {
part, err := writer.CreateFormFile("file", file.name)
if err != nil {
t.Fatalf("failed to create multipart file: %v", err)
}
if _, err = part.Write([]byte(file.content)); err != nil {
t.Fatalf("failed to write multipart content: %v", err)
}
}
if err := writer.Close(); err != nil {
t.Fatalf("failed to close multipart writer: %v", err)
}
rec := httptest.NewRecorder()
ctx, _ := gin.CreateTestContext(rec)
req := httptest.NewRequest(http.MethodPost, "/v0/management/auth-files", &body)
req.Header.Set("Content-Type", writer.FormDataContentType())
ctx.Request = req
h.UploadAuthFile(ctx)
if rec.Code != http.StatusMultiStatus {
t.Fatalf("expected upload status %d, got %d with body %s", http.StatusMultiStatus, rec.Code, rec.Body.String())
}
data, err := os.ReadFile(filepath.Join(authDir, existingName))
if err != nil {
t.Fatalf("expected existing auth file to remain readable: %v", err)
}
if string(data) != existingContent {
t.Fatalf("expected existing auth file to remain %q, got %q", existingContent, string(data))
}
betaData, err := os.ReadFile(filepath.Join(authDir, "beta.json"))
if err != nil {
t.Fatalf("expected valid auth file to be created: %v", err)
}
if string(betaData) != files[1].content {
t.Fatalf("expected beta auth file content %q, got %q", files[1].content, string(betaData))
}
}
func TestDeleteAuthFile_BatchQuery(t *testing.T) {
t.Setenv("MANAGEMENT_PASSWORD", "")
gin.SetMode(gin.TestMode)
authDir := t.TempDir()
files := []string{"alpha.json", "beta.json"}
for _, name := range files {
if err := os.WriteFile(filepath.Join(authDir, name), []byte(`{"type":"codex"}`), 0o600); err != nil {
t.Fatalf("failed to write auth file %s: %v", name, err)
}
}
manager := coreauth.NewManager(nil, nil, nil)
h := NewHandlerWithoutConfigFilePath(&config.Config{AuthDir: authDir}, manager)
h.tokenStore = &memoryAuthStore{}
rec := httptest.NewRecorder()
ctx, _ := gin.CreateTestContext(rec)
req := httptest.NewRequest(
http.MethodDelete,
"/v0/management/auth-files?name="+url.QueryEscape(files[0])+"&name="+url.QueryEscape(files[1]),
nil,
)
ctx.Request = req
h.DeleteAuthFile(ctx)
if rec.Code != http.StatusOK {
t.Fatalf("expected delete status %d, got %d with body %s", http.StatusOK, rec.Code, rec.Body.String())
}
var payload map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &payload); err != nil {
t.Fatalf("failed to decode response: %v", err)
}
if got, ok := payload["deleted"].(float64); !ok || int(got) != len(files) {
t.Fatalf("expected deleted=%d, got %#v", len(files), payload["deleted"])
}
for _, name := range files {
if _, err := os.Stat(filepath.Join(authDir, name)); !os.IsNotExist(err) {
t.Fatalf("expected auth file %s to be removed, stat err: %v", name, err)
}
}
}
@@ -0,0 +1,62 @@
package management
import (
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"testing"
"github.com/gin-gonic/gin"
"github.com/router-for-me/CLIProxyAPI/v6/internal/config"
)
func TestDownloadAuthFile_ReturnsFile(t *testing.T) {
t.Setenv("MANAGEMENT_PASSWORD", "")
gin.SetMode(gin.TestMode)
authDir := t.TempDir()
fileName := "download-user.json"
expected := []byte(`{"type":"codex"}`)
if err := os.WriteFile(filepath.Join(authDir, fileName), expected, 0o600); err != nil {
t.Fatalf("failed to write auth file: %v", err)
}
h := NewHandlerWithoutConfigFilePath(&config.Config{AuthDir: authDir}, nil)
rec := httptest.NewRecorder()
ctx, _ := gin.CreateTestContext(rec)
ctx.Request = httptest.NewRequest(http.MethodGet, "/v0/management/auth-files/download?name="+url.QueryEscape(fileName), nil)
h.DownloadAuthFile(ctx)
if rec.Code != http.StatusOK {
t.Fatalf("expected download status %d, got %d with body %s", http.StatusOK, rec.Code, rec.Body.String())
}
if got := rec.Body.Bytes(); string(got) != string(expected) {
t.Fatalf("unexpected download content: %q", string(got))
}
}
func TestDownloadAuthFile_RejectsPathSeparators(t *testing.T) {
t.Setenv("MANAGEMENT_PASSWORD", "")
gin.SetMode(gin.TestMode)
h := NewHandlerWithoutConfigFilePath(&config.Config{AuthDir: t.TempDir()}, nil)
for _, name := range []string{
"../external/secret.json",
`..\\external\\secret.json`,
"nested/secret.json",
`nested\\secret.json`,
} {
rec := httptest.NewRecorder()
ctx, _ := gin.CreateTestContext(rec)
ctx.Request = httptest.NewRequest(http.MethodGet, "/v0/management/auth-files/download?name="+url.QueryEscape(name), nil)
h.DownloadAuthFile(ctx)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected %d for name %q, got %d with body %s", http.StatusBadRequest, name, rec.Code, rec.Body.String())
}
}
}
@@ -0,0 +1,51 @@
//go:build windows
package management
import (
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"testing"
"github.com/gin-gonic/gin"
"github.com/router-for-me/CLIProxyAPI/v6/internal/config"
)
func TestDownloadAuthFile_PreventsWindowsSlashTraversal(t *testing.T) {
t.Setenv("MANAGEMENT_PASSWORD", "")
gin.SetMode(gin.TestMode)
tempDir := t.TempDir()
authDir := filepath.Join(tempDir, "auth")
externalDir := filepath.Join(tempDir, "external")
if err := os.MkdirAll(authDir, 0o700); err != nil {
t.Fatalf("failed to create auth dir: %v", err)
}
if err := os.MkdirAll(externalDir, 0o700); err != nil {
t.Fatalf("failed to create external dir: %v", err)
}
secretName := "secret.json"
secretPath := filepath.Join(externalDir, secretName)
if err := os.WriteFile(secretPath, []byte(`{"secret":true}`), 0o600); err != nil {
t.Fatalf("failed to write external file: %v", err)
}
h := NewHandlerWithoutConfigFilePath(&config.Config{AuthDir: authDir}, nil)
rec := httptest.NewRecorder()
ctx, _ := gin.CreateTestContext(rec)
ctx.Request = httptest.NewRequest(
http.MethodGet,
"/v0/management/auth-files/download?name="+url.QueryEscape("../external/"+secretName),
nil,
)
h.DownloadAuthFile(ctx)
if rec.Code != http.StatusBadRequest {
t.Fatalf("expected status %d, got %d with body %s", http.StatusBadRequest, rec.Code, rec.Body.String())
}
}
@@ -123,6 +123,10 @@ func (fh *FallbackHandler) WrapHandler(handler gin.HandlerFunc) gin.HandlerFunc
return
}
// Sanitize request body: remove thinking blocks with invalid signatures
// to prevent upstream API 400 errors
bodyBytes = SanitizeAmpRequestBody(bodyBytes)
// Restore the body for the handler to read
c.Request.Body = io.NopCloser(bytes.NewReader(bodyBytes))
@@ -259,10 +263,16 @@ func (fh *FallbackHandler) WrapHandler(handler gin.HandlerFunc) gin.HandlerFunc
} else if len(providers) > 0 {
// Log: Using local provider (free)
logAmpRouting(RouteTypeLocalProvider, modelName, resolvedModel, providerName, requestPath)
// Wrap with ResponseRewriter for local providers too, because upstream
// proxies (e.g. NewAPI) may return a different model name and lack
// Amp-required fields like thinking.signature.
rewriter := NewResponseRewriter(c.Writer, modelName)
c.Writer = rewriter
// Filter Anthropic-Beta header only for local handling paths
filterAntropicBetaHeader(c)
c.Request.Body = io.NopCloser(bytes.NewReader(bodyBytes))
handler(c)
rewriter.Flush()
} else {
// No provider, no mapping, no proxy: fall back to the wrapped handler so it can return an error response
c.Request.Body = io.NopCloser(bytes.NewReader(bodyBytes))
+290 -35
View File
@@ -2,6 +2,7 @@ package amp
import (
"bytes"
"fmt"
"net/http"
"strings"
@@ -12,32 +13,83 @@ import (
)
// ResponseRewriter wraps a gin.ResponseWriter to intercept and modify the response body
// It's used to rewrite model names in responses when model mapping is used
// It is used to rewrite model names in responses when model mapping is used
// and to keep Amp-compatible response shapes.
type ResponseRewriter struct {
gin.ResponseWriter
body *bytes.Buffer
originalModel string
isStreaming bool
body *bytes.Buffer
originalModel string
isStreaming bool
suppressedContentBlock map[int]struct{}
}
// NewResponseRewriter creates a new response rewriter for model name substitution
// NewResponseRewriter creates a new response rewriter for model name substitution.
func NewResponseRewriter(w gin.ResponseWriter, originalModel string) *ResponseRewriter {
return &ResponseRewriter{
ResponseWriter: w,
body: &bytes.Buffer{},
originalModel: originalModel,
ResponseWriter: w,
body: &bytes.Buffer{},
originalModel: originalModel,
suppressedContentBlock: make(map[int]struct{}),
}
}
// Write intercepts response writes and buffers them for model name replacement
const maxBufferedResponseBytes = 2 * 1024 * 1024 // 2MB safety cap
func looksLikeSSEChunk(data []byte) bool {
for _, line := range bytes.Split(data, []byte("\n")) {
trimmed := bytes.TrimSpace(line)
if bytes.HasPrefix(trimmed, []byte("data:")) ||
bytes.HasPrefix(trimmed, []byte("event:")) {
return true
}
}
return false
}
func (rw *ResponseRewriter) enableStreaming(reason string) error {
if rw.isStreaming {
return nil
}
rw.isStreaming = true
if rw.body != nil && rw.body.Len() > 0 {
buf := rw.body.Bytes()
toFlush := make([]byte, len(buf))
copy(toFlush, buf)
rw.body.Reset()
if _, err := rw.ResponseWriter.Write(rw.rewriteStreamChunk(toFlush)); err != nil {
return err
}
if flusher, ok := rw.ResponseWriter.(http.Flusher); ok {
flusher.Flush()
}
}
log.Debugf("amp response rewriter: switched to streaming (%s)", reason)
return nil
}
func (rw *ResponseRewriter) Write(data []byte) (int, error) {
// Detect streaming on first write
if rw.body.Len() == 0 && !rw.isStreaming {
if !rw.isStreaming && rw.body.Len() == 0 {
contentType := rw.Header().Get("Content-Type")
rw.isStreaming = strings.Contains(contentType, "text/event-stream") ||
strings.Contains(contentType, "stream")
}
if !rw.isStreaming {
if looksLikeSSEChunk(data) {
if err := rw.enableStreaming("sse heuristic"); err != nil {
return 0, err
}
} else if rw.body.Len()+len(data) > maxBufferedResponseBytes {
log.Warnf("amp response rewriter: buffer exceeded %d bytes, switching to streaming", maxBufferedResponseBytes)
if err := rw.enableStreaming("buffer limit"); err != nil {
return 0, err
}
}
}
if rw.isStreaming {
n, err := rw.ResponseWriter.Write(rw.rewriteStreamChunk(data))
if err == nil {
@@ -50,7 +102,6 @@ func (rw *ResponseRewriter) Write(data []byte) (int, error) {
return rw.body.Write(data)
}
// Flush writes the buffered response with model names rewritten
func (rw *ResponseRewriter) Flush() {
if rw.isStreaming {
if flusher, ok := rw.ResponseWriter.(http.Flusher); ok {
@@ -59,26 +110,68 @@ func (rw *ResponseRewriter) Flush() {
return
}
if rw.body.Len() > 0 {
if _, err := rw.ResponseWriter.Write(rw.rewriteModelInResponse(rw.body.Bytes())); err != nil {
rewritten := rw.rewriteModelInResponse(rw.body.Bytes())
// Update Content-Length to match the rewritten body size, since
// signature injection and model name changes alter the payload length.
rw.ResponseWriter.Header().Set("Content-Length", fmt.Sprintf("%d", len(rewritten)))
if _, err := rw.ResponseWriter.Write(rewritten); err != nil {
log.Warnf("amp response rewriter: failed to write rewritten response: %v", err)
}
}
}
// modelFieldPaths lists all JSON paths where model name may appear
var modelFieldPaths = []string{"message.model", "model", "modelVersion", "response.model", "response.modelVersion"}
// rewriteModelInResponse replaces all occurrences of the mapped model with the original model in JSON
// It also suppresses "thinking" blocks if "tool_use" is present to ensure Amp client compatibility
func (rw *ResponseRewriter) rewriteModelInResponse(data []byte) []byte {
// 1. Amp Compatibility: Suppress thinking blocks if tool use is detected
// The Amp client struggles when both thinking and tool_use blocks are present
// ensureAmpSignature injects empty signature fields into tool_use/thinking blocks
// in API responses so that the Amp TUI does not crash on P.signature.length.
func ensureAmpSignature(data []byte) []byte {
for index, block := range gjson.GetBytes(data, "content").Array() {
blockType := block.Get("type").String()
if blockType != "tool_use" && blockType != "thinking" {
continue
}
signaturePath := fmt.Sprintf("content.%d.signature", index)
if gjson.GetBytes(data, signaturePath).Exists() {
continue
}
var err error
data, err = sjson.SetBytes(data, signaturePath, "")
if err != nil {
log.Warnf("Amp ResponseRewriter: failed to add empty signature to %s block: %v", blockType, err)
break
}
}
contentBlockType := gjson.GetBytes(data, "content_block.type").String()
if (contentBlockType == "tool_use" || contentBlockType == "thinking") && !gjson.GetBytes(data, "content_block.signature").Exists() {
var err error
data, err = sjson.SetBytes(data, "content_block.signature", "")
if err != nil {
log.Warnf("Amp ResponseRewriter: failed to add empty signature to streaming %s block: %v", contentBlockType, err)
}
}
return data
}
func (rw *ResponseRewriter) markSuppressedContentBlock(index int) {
if rw.suppressedContentBlock == nil {
rw.suppressedContentBlock = make(map[int]struct{})
}
rw.suppressedContentBlock[index] = struct{}{}
}
func (rw *ResponseRewriter) isSuppressedContentBlock(index int) bool {
_, ok := rw.suppressedContentBlock[index]
return ok
}
func (rw *ResponseRewriter) suppressAmpThinking(data []byte) []byte {
if gjson.GetBytes(data, `content.#(type=="tool_use")`).Exists() {
filtered := gjson.GetBytes(data, `content.#(type!="thinking")#`)
if filtered.Exists() {
originalCount := gjson.GetBytes(data, "content.#").Int()
filteredCount := filtered.Get("#").Int()
if originalCount > filteredCount {
var err error
data, err = sjson.SetBytes(data, "content", filtered.Value())
@@ -86,13 +179,41 @@ func (rw *ResponseRewriter) rewriteModelInResponse(data []byte) []byte {
log.Warnf("Amp ResponseRewriter: failed to suppress thinking blocks: %v", err)
} else {
log.Debugf("Amp ResponseRewriter: Suppressed %d thinking blocks due to tool usage", originalCount-filteredCount)
// Log the result for verification
log.Debugf("Amp ResponseRewriter: Resulting content: %s", gjson.GetBytes(data, "content").String())
}
}
}
}
eventType := gjson.GetBytes(data, "type").String()
indexResult := gjson.GetBytes(data, "index")
if eventType == "content_block_start" && gjson.GetBytes(data, "content_block.type").String() == "thinking" && indexResult.Exists() {
rw.markSuppressedContentBlock(int(indexResult.Int()))
return nil
}
if gjson.GetBytes(data, "delta.type").String() == "thinking_delta" {
if indexResult.Exists() {
rw.markSuppressedContentBlock(int(indexResult.Int()))
}
return nil
}
if eventType == "content_block_stop" && indexResult.Exists() {
index := int(indexResult.Int())
if rw.isSuppressedContentBlock(index) {
delete(rw.suppressedContentBlock, index)
return nil
}
}
return data
}
func (rw *ResponseRewriter) rewriteModelInResponse(data []byte) []byte {
data = ensureAmpSignature(data)
data = rw.suppressAmpThinking(data)
if len(data) == 0 {
return data
}
if rw.originalModel == "" {
return data
}
@@ -104,24 +225,158 @@ func (rw *ResponseRewriter) rewriteModelInResponse(data []byte) []byte {
return data
}
// rewriteStreamChunk rewrites model names in SSE stream chunks
func (rw *ResponseRewriter) rewriteStreamChunk(chunk []byte) []byte {
if rw.originalModel == "" {
return chunk
lines := bytes.Split(chunk, []byte("\n"))
var out [][]byte
i := 0
for i < len(lines) {
line := lines[i]
trimmed := bytes.TrimSpace(line)
// Case 1: "event:" line - look ahead for its "data:" line
if bytes.HasPrefix(trimmed, []byte("event: ")) {
// Scan forward past blank lines to find the data: line
dataIdx := -1
for j := i + 1; j < len(lines); j++ {
t := bytes.TrimSpace(lines[j])
if len(t) == 0 {
continue
}
if bytes.HasPrefix(t, []byte("data: ")) {
dataIdx = j
}
break
}
if dataIdx >= 0 {
// Found event+data pair - process through rewriter
jsonData := bytes.TrimPrefix(bytes.TrimSpace(lines[dataIdx]), []byte("data: "))
if len(jsonData) > 0 && jsonData[0] == '{' {
rewritten := rw.rewriteStreamEvent(jsonData)
if rewritten == nil {
// Event suppressed (e.g. thinking block), skip event+data pair
i = dataIdx + 1
continue
}
// Emit event line
out = append(out, line)
// Emit blank lines between event and data
for k := i + 1; k < dataIdx; k++ {
out = append(out, lines[k])
}
// Emit rewritten data
out = append(out, append([]byte("data: "), rewritten...))
i = dataIdx + 1
continue
}
}
// No data line found (orphan event from cross-chunk split)
// Pass it through as-is - the data will arrive in the next chunk
out = append(out, line)
i++
continue
}
// Case 2: standalone "data:" line (no preceding event: in this chunk)
if bytes.HasPrefix(trimmed, []byte("data: ")) {
jsonData := bytes.TrimPrefix(trimmed, []byte("data: "))
if len(jsonData) > 0 && jsonData[0] == '{' {
rewritten := rw.rewriteStreamEvent(jsonData)
if rewritten != nil {
out = append(out, append([]byte("data: "), rewritten...))
}
i++
continue
}
}
// Case 3: everything else
out = append(out, line)
i++
}
// SSE format: "data: {json}\n\n"
lines := bytes.Split(chunk, []byte("\n"))
for i, line := range lines {
if bytes.HasPrefix(line, []byte("data: ")) {
jsonData := bytes.TrimPrefix(line, []byte("data: "))
if len(jsonData) > 0 && jsonData[0] == '{' {
// Rewrite JSON in the data line
rewritten := rw.rewriteModelInResponse(jsonData)
lines[i] = append([]byte("data: "), rewritten...)
return bytes.Join(out, []byte("\n"))
}
// rewriteStreamEvent processes a single JSON event in the SSE stream.
// It rewrites model names and ensures signature fields exist.
func (rw *ResponseRewriter) rewriteStreamEvent(data []byte) []byte {
// Suppress thinking blocks before any other processing.
data = rw.suppressAmpThinking(data)
if len(data) == 0 {
return nil
}
// Inject empty signature where needed
data = ensureAmpSignature(data)
// Rewrite model name
if rw.originalModel != "" {
for _, path := range modelFieldPaths {
if gjson.GetBytes(data, path).Exists() {
data, _ = sjson.SetBytes(data, path, rw.originalModel)
}
}
}
return bytes.Join(lines, []byte("\n"))
return data
}
// SanitizeAmpRequestBody removes thinking blocks with empty/missing/invalid signatures
// from the messages array in a request body before forwarding to the upstream API.
// This prevents 400 errors from the API which requires valid signatures on thinking blocks.
func SanitizeAmpRequestBody(body []byte) []byte {
messages := gjson.GetBytes(body, "messages")
if !messages.Exists() || !messages.IsArray() {
return body
}
modified := false
for msgIdx, msg := range messages.Array() {
if msg.Get("role").String() != "assistant" {
continue
}
content := msg.Get("content")
if !content.Exists() || !content.IsArray() {
continue
}
var keepBlocks []interface{}
removedCount := 0
for _, block := range content.Array() {
blockType := block.Get("type").String()
if blockType == "thinking" {
sig := block.Get("signature")
if !sig.Exists() || sig.Type != gjson.String || strings.TrimSpace(sig.String()) == "" {
removedCount++
continue
}
}
keepBlocks = append(keepBlocks, block.Value())
}
if removedCount > 0 {
contentPath := fmt.Sprintf("messages.%d.content", msgIdx)
var err error
if len(keepBlocks) == 0 {
body, err = sjson.SetBytes(body, contentPath, []interface{}{})
} else {
body, err = sjson.SetBytes(body, contentPath, keepBlocks)
}
if err != nil {
log.Warnf("Amp RequestSanitizer: failed to remove thinking blocks from message %d: %v", msgIdx, err)
continue
}
modified = true
log.Debugf("Amp RequestSanitizer: removed %d thinking blocks with invalid signatures from message %d", removedCount, msgIdx)
}
}
if modified {
log.Debugf("Amp RequestSanitizer: sanitized request body")
}
return body
}
@@ -100,6 +100,44 @@ func TestRewriteStreamChunk_MessageModel(t *testing.T) {
}
}
func TestRewriteStreamChunk_SuppressesThinkingContentBlockFrames(t *testing.T) {
rw := &ResponseRewriter{suppressedContentBlock: make(map[int]struct{})}
chunk := []byte("event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"thinking\",\"thinking\":\"\"}}\n\nevent: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"thinking_delta\",\"thinking\":\"abc\"}}\n\nevent: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\nevent: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":1,\"content_block\":{\"type\":\"tool_use\",\"name\":\"bash\",\"input\":{}}}\n\n")
result := rw.rewriteStreamChunk(chunk)
if contains(result, []byte("\"thinking\"")) || contains(result, []byte("\"thinking_delta\"")) {
t.Fatalf("expected thinking content_block frames to be suppressed, got %s", string(result))
}
if contains(result, []byte("content_block_stop")) {
t.Fatalf("expected suppressed thinking content_block_stop to be removed, got %s", string(result))
}
if !contains(result, []byte("\"tool_use\"")) {
t.Fatalf("expected tool_use content_block frame to remain, got %s", string(result))
}
if !contains(result, []byte("\"signature\":\"\"")) {
t.Fatalf("expected tool_use content_block signature injection, got %s", string(result))
}
}
func TestSanitizeAmpRequestBody_RemovesWhitespaceAndNonStringSignatures(t *testing.T) {
input := []byte(`{"messages":[{"role":"assistant","content":[{"type":"thinking","thinking":"drop-whitespace","signature":" "},{"type":"thinking","thinking":"drop-number","signature":123},{"type":"thinking","thinking":"keep-valid","signature":"valid-signature"},{"type":"text","text":"keep-text"}]}]}`)
result := SanitizeAmpRequestBody(input)
if contains(result, []byte("drop-whitespace")) {
t.Fatalf("expected whitespace-only signature block to be removed, got %s", string(result))
}
if contains(result, []byte("drop-number")) {
t.Fatalf("expected non-string signature block to be removed, got %s", string(result))
}
if !contains(result, []byte("keep-valid")) {
t.Fatalf("expected valid thinking block to remain, got %s", string(result))
}
if !contains(result, []byte("keep-text")) {
t.Fatalf("expected non-thinking content to remain, got %s", string(result))
}
}
func contains(data, substr []byte) bool {
for i := 0; i <= len(data)-len(substr); i++ {
if string(data[i:i+len(substr)]) == string(substr) {
@@ -0,0 +1,55 @@
package config
import (
"os"
"path/filepath"
"testing"
)
func TestLoadConfigOptional_ClaudeHeaderDefaults(t *testing.T) {
dir := t.TempDir()
configPath := filepath.Join(dir, "config.yaml")
configYAML := []byte(`
claude-header-defaults:
user-agent: " claude-cli/2.1.70 (external, cli) "
package-version: " 0.80.0 "
runtime-version: " v24.5.0 "
os: " MacOS "
arch: " arm64 "
timeout: " 900 "
stabilize-device-profile: false
`)
if err := os.WriteFile(configPath, configYAML, 0o600); err != nil {
t.Fatalf("failed to write config: %v", err)
}
cfg, err := LoadConfigOptional(configPath, false)
if err != nil {
t.Fatalf("LoadConfigOptional() error = %v", err)
}
if got := cfg.ClaudeHeaderDefaults.UserAgent; got != "claude-cli/2.1.70 (external, cli)" {
t.Fatalf("UserAgent = %q, want %q", got, "claude-cli/2.1.70 (external, cli)")
}
if got := cfg.ClaudeHeaderDefaults.PackageVersion; got != "0.80.0" {
t.Fatalf("PackageVersion = %q, want %q", got, "0.80.0")
}
if got := cfg.ClaudeHeaderDefaults.RuntimeVersion; got != "v24.5.0" {
t.Fatalf("RuntimeVersion = %q, want %q", got, "v24.5.0")
}
if got := cfg.ClaudeHeaderDefaults.OS; got != "MacOS" {
t.Fatalf("OS = %q, want %q", got, "MacOS")
}
if got := cfg.ClaudeHeaderDefaults.Arch; got != "arm64" {
t.Fatalf("Arch = %q, want %q", got, "arm64")
}
if got := cfg.ClaudeHeaderDefaults.Timeout; got != "900" {
t.Fatalf("Timeout = %q, want %q", got, "900")
}
if cfg.ClaudeHeaderDefaults.StabilizeDeviceProfile == nil {
t.Fatal("StabilizeDeviceProfile = nil, want non-nil")
}
if got := *cfg.ClaudeHeaderDefaults.StabilizeDeviceProfile; got {
t.Fatalf("StabilizeDeviceProfile = %v, want false", got)
}
}
+37 -6
View File
@@ -13,6 +13,7 @@ import (
"strings"
"syscall"
"github.com/router-for-me/CLIProxyAPI/v6/internal/registry"
log "github.com/sirupsen/logrus"
"golang.org/x/crypto/bcrypt"
"gopkg.in/yaml.v3"
@@ -128,13 +129,19 @@ type Config struct {
legacyMigrationPending bool `yaml:"-" json:"-"`
}
// ClaudeHeaderDefaults configures default header values injected into Claude API requests
// when the client does not send them. Update these when Claude Code releases a new version.
// ClaudeHeaderDefaults configures default header values injected into Claude API requests.
// In legacy mode, UserAgent/PackageVersion/RuntimeVersion/Timeout act as fallbacks when
// the client omits them, while OS/Arch remain runtime-derived. When stabilized device
// profiles are enabled, OS/Arch become the pinned platform baseline, while
// UserAgent/PackageVersion/RuntimeVersion seed the upgradeable software fingerprint.
type ClaudeHeaderDefaults struct {
UserAgent string `yaml:"user-agent" json:"user-agent"`
PackageVersion string `yaml:"package-version" json:"package-version"`
RuntimeVersion string `yaml:"runtime-version" json:"runtime-version"`
Timeout string `yaml:"timeout" json:"timeout"`
UserAgent string `yaml:"user-agent" json:"user-agent"`
PackageVersion string `yaml:"package-version" json:"package-version"`
RuntimeVersion string `yaml:"runtime-version" json:"runtime-version"`
OS string `yaml:"os" json:"os"`
Arch string `yaml:"arch" json:"arch"`
Timeout string `yaml:"timeout" json:"timeout"`
StabilizeDeviceProfile *bool `yaml:"stabilize-device-profile,omitempty" json:"stabilize-device-profile,omitempty"`
}
// CodexHeaderDefaults configures fallback header values injected into Codex
@@ -171,6 +178,9 @@ type RemoteManagement struct {
SecretKey string `yaml:"secret-key"`
// DisableControlPanel skips serving and syncing the bundled management UI when true.
DisableControlPanel bool `yaml:"disable-control-panel"`
// DisableAutoUpdatePanel disables automatic periodic background updates of the management panel asset from GitHub.
// When false (the default), the background updater remains enabled; when true, the panel is only downloaded on first access if missing.
DisableAutoUpdatePanel bool `yaml:"disable-auto-update-panel"`
// PanelGitHubRepository overrides the GitHub repository used to fetch the management panel asset.
// Accepts either a repository URL (https://github.com/org/repo) or an API releases endpoint.
PanelGitHubRepository string `yaml:"panel-github-repository"`
@@ -505,6 +515,10 @@ type OpenAICompatibilityModel struct {
// Alias is the model name alias that clients will use to reference this model.
Alias string `yaml:"alias" json:"alias"`
// Thinking configures the thinking/reasoning capability for this model.
// If nil, the model defaults to level-based reasoning with levels ["low", "medium", "high"].
Thinking *registry.ThinkingSupport `yaml:"thinking,omitempty" json:"thinking,omitempty"`
}
func (m OpenAICompatibilityModel) GetName() string { return m.Name }
@@ -630,6 +644,9 @@ func LoadConfigOptional(configFile string, optional bool) (*Config, error) {
// Sanitize Codex header defaults.
cfg.SanitizeCodexHeaderDefaults()
// Sanitize Claude header defaults.
cfg.SanitizeClaudeHeaderDefaults()
// Sanitize Claude key headers
cfg.SanitizeClaudeKeys()
@@ -729,6 +746,20 @@ func (cfg *Config) SanitizeCodexHeaderDefaults() {
cfg.CodexHeaderDefaults.BetaFeatures = strings.TrimSpace(cfg.CodexHeaderDefaults.BetaFeatures)
}
// SanitizeClaudeHeaderDefaults trims surrounding whitespace from the
// configured Claude fingerprint baseline values.
func (cfg *Config) SanitizeClaudeHeaderDefaults() {
if cfg == nil {
return
}
cfg.ClaudeHeaderDefaults.UserAgent = strings.TrimSpace(cfg.ClaudeHeaderDefaults.UserAgent)
cfg.ClaudeHeaderDefaults.PackageVersion = strings.TrimSpace(cfg.ClaudeHeaderDefaults.PackageVersion)
cfg.ClaudeHeaderDefaults.RuntimeVersion = strings.TrimSpace(cfg.ClaudeHeaderDefaults.RuntimeVersion)
cfg.ClaudeHeaderDefaults.OS = strings.TrimSpace(cfg.ClaudeHeaderDefaults.OS)
cfg.ClaudeHeaderDefaults.Arch = strings.TrimSpace(cfg.ClaudeHeaderDefaults.Arch)
cfg.ClaudeHeaderDefaults.Timeout = strings.TrimSpace(cfg.ClaudeHeaderDefaults.Timeout)
}
// SanitizeOAuthModelAlias normalizes and deduplicates global OAuth model name aliases.
// It trims whitespace, normalizes channel keys to lower-case, drops empty entries,
// allows multiple aliases per upstream name, and ensures aliases are unique within each channel.
+14 -2
View File
@@ -31,6 +31,7 @@ const (
httpUserAgent = "CLIProxyAPI-management-updater"
managementSyncMinInterval = 30 * time.Second
updateCheckInterval = 3 * time.Hour
maxAssetDownloadSize = 50 << 20 // 10 MB safety limit for management asset downloads
)
// ManagementFileName exposes the control panel asset filename.
@@ -88,6 +89,10 @@ func runAutoUpdater(ctx context.Context) {
log.Debug("management asset auto-updater skipped: control panel disabled")
return
}
if cfg.RemoteManagement.DisableAutoUpdatePanel {
log.Debug("management asset auto-updater skipped: disable-auto-update-panel is enabled")
return
}
configPath, _ := schedulerConfigPath.Load().(string)
staticDir := StaticDir(configPath)
@@ -259,7 +264,8 @@ func EnsureLatestManagementHTML(ctx context.Context, staticDir string, proxyURL
}
if remoteHash != "" && !strings.EqualFold(remoteHash, downloadedHash) {
log.Warnf("remote digest mismatch for management asset: expected %s got %s", remoteHash, downloadedHash)
log.Errorf("management asset digest mismatch: expected %s got %s — aborting update for safety", remoteHash, downloadedHash)
return nil, nil
}
if err = atomicWriteFile(localPath, data); err != nil {
@@ -282,6 +288,9 @@ func ensureFallbackManagementHTML(ctx context.Context, client *http.Client, loca
return false
}
log.Warnf("management asset downloaded from fallback URL without digest verification (hash=%s) — "+
"enable verified GitHub updates by keeping disable-auto-update-panel set to false", downloadedHash)
if err = atomicWriteFile(localPath, data); err != nil {
log.WithError(err).Warn("failed to persist fallback management control panel page")
return false
@@ -392,10 +401,13 @@ func downloadAsset(ctx context.Context, client *http.Client, downloadURL string)
return nil, "", fmt.Errorf("unexpected download status %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
}
data, err := io.ReadAll(resp.Body)
data, err := io.ReadAll(io.LimitReader(resp.Body, maxAssetDownloadSize+1))
if err != nil {
return nil, "", fmt.Errorf("read download body: %w", err)
}
if int64(len(data)) > maxAssetDownloadSize {
return nil, "", fmt.Errorf("download exceeds maximum allowed size of %d bytes", maxAssetDownloadSize)
}
sum := sha256.Sum256(data)
return data, hex.EncodeToString(sum[:]), nil
+5 -5
View File
@@ -71,16 +71,16 @@ type availableModelsCacheEntry struct {
// Values are interpreted in provider-native token units.
type ThinkingSupport struct {
// Min is the minimum allowed thinking budget (inclusive).
Min int `json:"min,omitempty"`
Min int `json:"min,omitempty" yaml:"min,omitempty"`
// Max is the maximum allowed thinking budget (inclusive).
Max int `json:"max,omitempty"`
Max int `json:"max,omitempty" yaml:"max,omitempty"`
// ZeroAllowed indicates whether 0 is a valid value (to disable thinking).
ZeroAllowed bool `json:"zero_allowed,omitempty"`
ZeroAllowed bool `json:"zero_allowed,omitempty" yaml:"zero-allowed,omitempty"`
// DynamicAllowed indicates whether -1 is a valid value (dynamic thinking budget).
DynamicAllowed bool `json:"dynamic_allowed,omitempty"`
DynamicAllowed bool `json:"dynamic_allowed,omitempty" yaml:"dynamic-allowed,omitempty"`
// Levels defines discrete reasoning effort levels (e.g., "low", "medium", "high").
// When set, the model uses level-based reasoning instead of token budgets.
Levels []string `json:"levels,omitempty"`
Levels []string `json:"levels,omitempty" yaml:"levels,omitempty"`
}
// ModelRegistration tracks a model's availability
@@ -0,0 +1,383 @@
package executor
import (
"crypto/sha256"
"encoding/hex"
"net/http"
"regexp"
"runtime"
"strconv"
"strings"
"sync"
"time"
"github.com/router-for-me/CLIProxyAPI/v6/internal/config"
cliproxyauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
)
const (
defaultClaudeFingerprintUserAgent = "claude-cli/2.1.63 (external, cli)"
defaultClaudeFingerprintPackageVersion = "0.74.0"
defaultClaudeFingerprintRuntimeVersion = "v24.3.0"
defaultClaudeFingerprintOS = "MacOS"
defaultClaudeFingerprintArch = "arm64"
claudeDeviceProfileTTL = 7 * 24 * time.Hour
claudeDeviceProfileCleanupPeriod = time.Hour
)
var (
claudeCLIVersionPattern = regexp.MustCompile(`^claude-cli/(\d+)\.(\d+)\.(\d+)`)
claudeDeviceProfileCache = make(map[string]claudeDeviceProfileCacheEntry)
claudeDeviceProfileCacheMu sync.RWMutex
claudeDeviceProfileCacheCleanupOnce sync.Once
claudeDeviceProfileBeforeCandidateStore func(claudeDeviceProfile)
)
type claudeCLIVersion struct {
major int
minor int
patch int
}
func (v claudeCLIVersion) Compare(other claudeCLIVersion) int {
switch {
case v.major != other.major:
if v.major > other.major {
return 1
}
return -1
case v.minor != other.minor:
if v.minor > other.minor {
return 1
}
return -1
case v.patch != other.patch:
if v.patch > other.patch {
return 1
}
return -1
default:
return 0
}
}
type claudeDeviceProfile struct {
UserAgent string
PackageVersion string
RuntimeVersion string
OS string
Arch string
Version claudeCLIVersion
HasVersion bool
}
type claudeDeviceProfileCacheEntry struct {
profile claudeDeviceProfile
expire time.Time
}
func claudeDeviceProfileStabilizationEnabled(cfg *config.Config) bool {
if cfg == nil || cfg.ClaudeHeaderDefaults.StabilizeDeviceProfile == nil {
return false
}
return *cfg.ClaudeHeaderDefaults.StabilizeDeviceProfile
}
func defaultClaudeDeviceProfile(cfg *config.Config) claudeDeviceProfile {
hdrDefault := func(cfgVal, fallback string) string {
if strings.TrimSpace(cfgVal) != "" {
return strings.TrimSpace(cfgVal)
}
return fallback
}
var hd config.ClaudeHeaderDefaults
if cfg != nil {
hd = cfg.ClaudeHeaderDefaults
}
profile := claudeDeviceProfile{
UserAgent: hdrDefault(hd.UserAgent, defaultClaudeFingerprintUserAgent),
PackageVersion: hdrDefault(hd.PackageVersion, defaultClaudeFingerprintPackageVersion),
RuntimeVersion: hdrDefault(hd.RuntimeVersion, defaultClaudeFingerprintRuntimeVersion),
OS: hdrDefault(hd.OS, defaultClaudeFingerprintOS),
Arch: hdrDefault(hd.Arch, defaultClaudeFingerprintArch),
}
if version, ok := parseClaudeCLIVersion(profile.UserAgent); ok {
profile.Version = version
profile.HasVersion = true
}
return profile
}
// mapStainlessOS maps runtime.GOOS to Stainless SDK OS names.
func mapStainlessOS() string {
switch runtime.GOOS {
case "darwin":
return "MacOS"
case "windows":
return "Windows"
case "linux":
return "Linux"
case "freebsd":
return "FreeBSD"
default:
return "Other::" + runtime.GOOS
}
}
// mapStainlessArch maps runtime.GOARCH to Stainless SDK architecture names.
func mapStainlessArch() string {
switch runtime.GOARCH {
case "amd64":
return "x64"
case "arm64":
return "arm64"
case "386":
return "x86"
default:
return "other::" + runtime.GOARCH
}
}
func parseClaudeCLIVersion(userAgent string) (claudeCLIVersion, bool) {
matches := claudeCLIVersionPattern.FindStringSubmatch(strings.TrimSpace(userAgent))
if len(matches) != 4 {
return claudeCLIVersion{}, false
}
major, err := strconv.Atoi(matches[1])
if err != nil {
return claudeCLIVersion{}, false
}
minor, err := strconv.Atoi(matches[2])
if err != nil {
return claudeCLIVersion{}, false
}
patch, err := strconv.Atoi(matches[3])
if err != nil {
return claudeCLIVersion{}, false
}
return claudeCLIVersion{major: major, minor: minor, patch: patch}, true
}
func shouldUpgradeClaudeDeviceProfile(candidate, current claudeDeviceProfile) bool {
if candidate.UserAgent == "" || !candidate.HasVersion {
return false
}
if current.UserAgent == "" || !current.HasVersion {
return true
}
return candidate.Version.Compare(current.Version) > 0
}
func pinClaudeDeviceProfilePlatform(profile, baseline claudeDeviceProfile) claudeDeviceProfile {
profile.OS = baseline.OS
profile.Arch = baseline.Arch
return profile
}
// normalizeClaudeDeviceProfile keeps stabilized profiles pinned to the current
// baseline platform and enforces the baseline software fingerprint as a floor.
func normalizeClaudeDeviceProfile(profile, baseline claudeDeviceProfile) claudeDeviceProfile {
profile = pinClaudeDeviceProfilePlatform(profile, baseline)
if profile.UserAgent == "" || !profile.HasVersion || shouldUpgradeClaudeDeviceProfile(baseline, profile) {
profile.UserAgent = baseline.UserAgent
profile.PackageVersion = baseline.PackageVersion
profile.RuntimeVersion = baseline.RuntimeVersion
profile.Version = baseline.Version
profile.HasVersion = baseline.HasVersion
}
return profile
}
func extractClaudeDeviceProfile(headers http.Header, cfg *config.Config) (claudeDeviceProfile, bool) {
if headers == nil {
return claudeDeviceProfile{}, false
}
userAgent := strings.TrimSpace(headers.Get("User-Agent"))
version, ok := parseClaudeCLIVersion(userAgent)
if !ok {
return claudeDeviceProfile{}, false
}
baseline := defaultClaudeDeviceProfile(cfg)
profile := claudeDeviceProfile{
UserAgent: userAgent,
PackageVersion: firstNonEmptyHeader(headers, "X-Stainless-Package-Version", baseline.PackageVersion),
RuntimeVersion: firstNonEmptyHeader(headers, "X-Stainless-Runtime-Version", baseline.RuntimeVersion),
OS: firstNonEmptyHeader(headers, "X-Stainless-Os", baseline.OS),
Arch: firstNonEmptyHeader(headers, "X-Stainless-Arch", baseline.Arch),
Version: version,
HasVersion: true,
}
return profile, true
}
func firstNonEmptyHeader(headers http.Header, name, fallback string) string {
if headers == nil {
return fallback
}
if value := strings.TrimSpace(headers.Get(name)); value != "" {
return value
}
return fallback
}
func claudeDeviceProfileScopeKey(auth *cliproxyauth.Auth, apiKey string) string {
switch {
case auth != nil && strings.TrimSpace(auth.ID) != "":
return "auth:" + strings.TrimSpace(auth.ID)
case strings.TrimSpace(apiKey) != "":
return "api_key:" + strings.TrimSpace(apiKey)
default:
return "global"
}
}
func claudeDeviceProfileCacheKey(auth *cliproxyauth.Auth, apiKey string) string {
sum := sha256.Sum256([]byte(claudeDeviceProfileScopeKey(auth, apiKey)))
return hex.EncodeToString(sum[:])
}
func startClaudeDeviceProfileCacheCleanup() {
go func() {
ticker := time.NewTicker(claudeDeviceProfileCleanupPeriod)
defer ticker.Stop()
for range ticker.C {
purgeExpiredClaudeDeviceProfiles()
}
}()
}
func purgeExpiredClaudeDeviceProfiles() {
now := time.Now()
claudeDeviceProfileCacheMu.Lock()
for key, entry := range claudeDeviceProfileCache {
if !entry.expire.After(now) {
delete(claudeDeviceProfileCache, key)
}
}
claudeDeviceProfileCacheMu.Unlock()
}
func resolveClaudeDeviceProfile(auth *cliproxyauth.Auth, apiKey string, headers http.Header, cfg *config.Config) claudeDeviceProfile {
claudeDeviceProfileCacheCleanupOnce.Do(startClaudeDeviceProfileCacheCleanup)
cacheKey := claudeDeviceProfileCacheKey(auth, apiKey)
now := time.Now()
baseline := defaultClaudeDeviceProfile(cfg)
candidate, hasCandidate := extractClaudeDeviceProfile(headers, cfg)
if hasCandidate {
candidate = pinClaudeDeviceProfilePlatform(candidate, baseline)
}
if hasCandidate && !shouldUpgradeClaudeDeviceProfile(candidate, baseline) {
hasCandidate = false
}
claudeDeviceProfileCacheMu.RLock()
entry, hasCached := claudeDeviceProfileCache[cacheKey]
cachedValid := hasCached && entry.expire.After(now) && entry.profile.UserAgent != ""
claudeDeviceProfileCacheMu.RUnlock()
if hasCandidate {
if claudeDeviceProfileBeforeCandidateStore != nil {
claudeDeviceProfileBeforeCandidateStore(candidate)
}
claudeDeviceProfileCacheMu.Lock()
entry, hasCached = claudeDeviceProfileCache[cacheKey]
cachedValid = hasCached && entry.expire.After(now) && entry.profile.UserAgent != ""
if cachedValid {
entry.profile = normalizeClaudeDeviceProfile(entry.profile, baseline)
}
if cachedValid && !shouldUpgradeClaudeDeviceProfile(candidate, entry.profile) {
entry.expire = now.Add(claudeDeviceProfileTTL)
claudeDeviceProfileCache[cacheKey] = entry
claudeDeviceProfileCacheMu.Unlock()
return entry.profile
}
claudeDeviceProfileCache[cacheKey] = claudeDeviceProfileCacheEntry{
profile: candidate,
expire: now.Add(claudeDeviceProfileTTL),
}
claudeDeviceProfileCacheMu.Unlock()
return candidate
}
if cachedValid {
claudeDeviceProfileCacheMu.Lock()
entry = claudeDeviceProfileCache[cacheKey]
if entry.expire.After(now) && entry.profile.UserAgent != "" {
entry.profile = normalizeClaudeDeviceProfile(entry.profile, baseline)
entry.expire = now.Add(claudeDeviceProfileTTL)
claudeDeviceProfileCache[cacheKey] = entry
claudeDeviceProfileCacheMu.Unlock()
return entry.profile
}
claudeDeviceProfileCacheMu.Unlock()
}
return baseline
}
func applyClaudeDeviceProfileHeaders(r *http.Request, profile claudeDeviceProfile) {
if r == nil {
return
}
for _, headerName := range []string{
"User-Agent",
"X-Stainless-Package-Version",
"X-Stainless-Runtime-Version",
"X-Stainless-Os",
"X-Stainless-Arch",
} {
r.Header.Del(headerName)
}
r.Header.Set("User-Agent", profile.UserAgent)
r.Header.Set("X-Stainless-Package-Version", profile.PackageVersion)
r.Header.Set("X-Stainless-Runtime-Version", profile.RuntimeVersion)
r.Header.Set("X-Stainless-Os", profile.OS)
r.Header.Set("X-Stainless-Arch", profile.Arch)
}
func applyClaudeLegacyDeviceHeaders(r *http.Request, ginHeaders http.Header, cfg *config.Config) {
if r == nil {
return
}
profile := defaultClaudeDeviceProfile(cfg)
miscEnsure := func(name, fallback string) {
if strings.TrimSpace(r.Header.Get(name)) != "" {
return
}
if strings.TrimSpace(ginHeaders.Get(name)) != "" {
r.Header.Set(name, strings.TrimSpace(ginHeaders.Get(name)))
return
}
r.Header.Set(name, fallback)
}
miscEnsure("X-Stainless-Runtime-Version", profile.RuntimeVersion)
miscEnsure("X-Stainless-Package-Version", profile.PackageVersion)
miscEnsure("X-Stainless-Os", mapStainlessOS())
miscEnsure("X-Stainless-Arch", mapStainlessArch())
// Legacy mode preserves per-auth custom header overrides. By the time we get
// here, ApplyCustomHeadersFromAttrs has already populated r.Header.
if strings.TrimSpace(r.Header.Get("User-Agent")) != "" {
return
}
clientUA := ""
if ginHeaders != nil {
clientUA = strings.TrimSpace(ginHeaders.Get("User-Agent"))
}
if isClaudeCodeClient(clientUA) {
r.Header.Set("User-Agent", clientUA)
return
}
r.Header.Set("User-Agent", profile.UserAgent)
}
+43 -49
View File
@@ -14,7 +14,6 @@ import (
"io"
"net/http"
"net/textproto"
"runtime"
"strings"
"time"
@@ -23,6 +22,7 @@ import (
claudeauth "github.com/router-for-me/CLIProxyAPI/v6/internal/auth/claude"
"github.com/router-for-me/CLIProxyAPI/v6/internal/config"
"github.com/router-for-me/CLIProxyAPI/v6/internal/misc"
"github.com/router-for-me/CLIProxyAPI/v6/internal/registry"
"github.com/router-for-me/CLIProxyAPI/v6/internal/thinking"
"github.com/router-for-me/CLIProxyAPI/v6/internal/util"
cliproxyauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
@@ -45,6 +45,10 @@ type ClaudeExecutor struct {
// Previously "proxy_" was used but this is a detectable fingerprint difference.
const claudeToolPrefix = ""
// Anthropic-compatible upstreams may reject or even crash when Claude models
// omit max_tokens. Prefer registered model metadata before using a fallback.
const defaultModelMaxTokens = 1024
func NewClaudeExecutor(cfg *config.Config) *ClaudeExecutor { return &ClaudeExecutor{cfg: cfg} }
func (e *ClaudeExecutor) Identifier() string { return "claude" }
@@ -128,6 +132,7 @@ func (e *ClaudeExecutor) Execute(ctx context.Context, auth *cliproxyauth.Auth, r
requestedModel := payloadRequestedModel(opts, req.Model)
body = applyPayloadConfigWithRoot(e.cfg, baseModel, to.String(), "", body, originalTranslated, requestedModel)
body = ensureModelMaxTokens(body, baseModel)
// Disable thinking if tool_choice forces tool use (Anthropic API constraint)
body = disableThinkingIfToolChoiceForced(body)
@@ -294,6 +299,7 @@ func (e *ClaudeExecutor) ExecuteStream(ctx context.Context, auth *cliproxyauth.A
requestedModel := payloadRequestedModel(opts, req.Model)
body = applyPayloadConfigWithRoot(e.cfg, baseModel, to.String(), "", body, originalTranslated, requestedModel)
body = ensureModelMaxTokens(body, baseModel)
// Disable thinking if tool_choice forces tool use (Anthropic API constraint)
body = disableThinkingIfToolChoiceForced(body)
@@ -767,36 +773,6 @@ func decodeResponseBody(body io.ReadCloser, contentEncoding string) (io.ReadClos
return body, nil
}
// mapStainlessOS maps runtime.GOOS to Stainless SDK OS names.
func mapStainlessOS() string {
switch runtime.GOOS {
case "darwin":
return "MacOS"
case "windows":
return "Windows"
case "linux":
return "Linux"
case "freebsd":
return "FreeBSD"
default:
return "Other::" + runtime.GOOS
}
}
// mapStainlessArch maps runtime.GOARCH to Stainless SDK architecture names.
func mapStainlessArch() string {
switch runtime.GOARCH {
case "amd64":
return "x64"
case "arm64":
return "arm64"
case "386":
return "x86"
default:
return "other::" + runtime.GOARCH
}
}
func applyClaudeHeaders(r *http.Request, auth *cliproxyauth.Auth, apiKey string, stream bool, extraBetas []string, cfg *config.Config) {
hdrDefault := func(cfgVal, fallback string) string {
if cfgVal != "" {
@@ -824,6 +800,11 @@ func applyClaudeHeaders(r *http.Request, auth *cliproxyauth.Auth, apiKey string,
if ginCtx, ok := r.Context().Value("gin").(*gin.Context); ok && ginCtx != nil && ginCtx.Request != nil {
ginHeaders = ginCtx.Request.Header
}
stabilizeDeviceProfile := claudeDeviceProfileStabilizationEnabled(cfg)
var deviceProfile claudeDeviceProfile
if stabilizeDeviceProfile {
deviceProfile = resolveClaudeDeviceProfile(auth, apiKey, ginHeaders, cfg)
}
baseBetas := "claude-code-20250219,oauth-2025-04-20,interleaved-thinking-2025-05-14,context-management-2025-06-27,prompt-caching-scope-2026-01-05"
if val := strings.TrimSpace(ginHeaders.Get("Anthropic-Beta")); val != "" {
@@ -867,25 +848,9 @@ func applyClaudeHeaders(r *http.Request, auth *cliproxyauth.Auth, apiKey string,
misc.EnsureHeader(r.Header, ginHeaders, "X-App", "cli")
// Values below match Claude Code 2.1.63 / @anthropic-ai/sdk 0.74.0 (updated 2026-02-28).
misc.EnsureHeader(r.Header, ginHeaders, "X-Stainless-Retry-Count", "0")
misc.EnsureHeader(r.Header, ginHeaders, "X-Stainless-Runtime-Version", hdrDefault(hd.RuntimeVersion, "v24.3.0"))
misc.EnsureHeader(r.Header, ginHeaders, "X-Stainless-Package-Version", hdrDefault(hd.PackageVersion, "0.74.0"))
misc.EnsureHeader(r.Header, ginHeaders, "X-Stainless-Runtime", "node")
misc.EnsureHeader(r.Header, ginHeaders, "X-Stainless-Lang", "js")
misc.EnsureHeader(r.Header, ginHeaders, "X-Stainless-Arch", mapStainlessArch())
misc.EnsureHeader(r.Header, ginHeaders, "X-Stainless-Os", mapStainlessOS())
misc.EnsureHeader(r.Header, ginHeaders, "X-Stainless-Timeout", hdrDefault(hd.Timeout, "600"))
// For User-Agent, only forward the client's header if it's already a Claude Code client.
// Non-Claude-Code clients (e.g. curl, OpenAI SDKs) get the default Claude Code User-Agent
// to avoid leaking the real client identity during cloaking.
clientUA := ""
if ginHeaders != nil {
clientUA = ginHeaders.Get("User-Agent")
}
if isClaudeCodeClient(clientUA) {
r.Header.Set("User-Agent", clientUA)
} else {
r.Header.Set("User-Agent", hdrDefault(hd.UserAgent, "claude-cli/2.1.63 (external, cli)"))
}
r.Header.Set("Connection", "keep-alive")
if stream {
r.Header.Set("Accept", "text/event-stream")
@@ -897,13 +862,19 @@ func applyClaudeHeaders(r *http.Request, auth *cliproxyauth.Auth, apiKey string,
r.Header.Set("Accept", "application/json")
r.Header.Set("Accept-Encoding", "gzip, deflate, br, zstd")
}
// Keep OS/Arch mapping dynamic (not configurable).
// They intentionally continue to derive from runtime.GOOS/runtime.GOARCH.
// Legacy mode keeps OS/Arch runtime-derived; stabilized mode pins OS/Arch
// to the configured baseline while still allowing newer official
// User-Agent/package/runtime tuples to upgrade the software fingerprint.
var attrs map[string]string
if auth != nil {
attrs = auth.Attributes
}
util.ApplyCustomHeadersFromAttrs(r, attrs)
if stabilizeDeviceProfile {
applyClaudeDeviceProfileHeaders(r, deviceProfile)
} else {
applyClaudeLegacyDeviceHeaders(r, ginHeaders, cfg)
}
// Re-enforce Accept-Encoding: identity after ApplyCustomHeadersFromAttrs, which
// may override it with a user-configured value. Compressed SSE breaks the line
// scanner regardless of user preference, so this is non-negotiable for streams.
@@ -1916,3 +1887,26 @@ func injectSystemCacheControl(payload []byte) []byte {
return payload
}
func ensureModelMaxTokens(body []byte, modelID string) []byte {
if len(body) == 0 || !gjson.ValidBytes(body) {
return body
}
if maxTokens := gjson.GetBytes(body, "max_tokens"); maxTokens.Exists() {
return body
}
for _, provider := range registry.GetGlobalRegistry().GetModelProviders(strings.TrimSpace(modelID)) {
if strings.EqualFold(provider, "claude") {
maxTokens := defaultModelMaxTokens
if info := registry.GetGlobalRegistry().GetModelInfo(strings.TrimSpace(modelID), "claude"); info != nil && info.MaxCompletionTokens > 0 {
maxTokens = info.MaxCompletionTokens
}
body, _ = sjson.SetBytes(body, "max_tokens", maxTokens)
return body
}
}
return body
}
@@ -8,10 +8,14 @@ import (
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
"github.com/gin-gonic/gin"
"github.com/klauspost/compress/zstd"
"github.com/router-for-me/CLIProxyAPI/v6/internal/config"
"github.com/router-for-me/CLIProxyAPI/v6/internal/registry"
cliproxyauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
cliproxyexecutor "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/executor"
sdktranslator "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
@@ -19,6 +23,587 @@ import (
"github.com/tidwall/sjson"
)
func resetClaudeDeviceProfileCache() {
claudeDeviceProfileCacheMu.Lock()
claudeDeviceProfileCache = make(map[string]claudeDeviceProfileCacheEntry)
claudeDeviceProfileCacheMu.Unlock()
}
func newClaudeHeaderTestRequest(t *testing.T, incoming http.Header) *http.Request {
t.Helper()
gin.SetMode(gin.TestMode)
recorder := httptest.NewRecorder()
ginCtx, _ := gin.CreateTestContext(recorder)
ginReq := httptest.NewRequest(http.MethodPost, "http://localhost/v1/messages", nil)
ginReq.Header = incoming.Clone()
ginCtx.Request = ginReq
req := httptest.NewRequest(http.MethodPost, "https://api.anthropic.com/v1/messages", nil)
return req.WithContext(context.WithValue(req.Context(), "gin", ginCtx))
}
func assertClaudeFingerprint(t *testing.T, headers http.Header, userAgent, pkgVersion, runtimeVersion, osName, arch string) {
t.Helper()
if got := headers.Get("User-Agent"); got != userAgent {
t.Fatalf("User-Agent = %q, want %q", got, userAgent)
}
if got := headers.Get("X-Stainless-Package-Version"); got != pkgVersion {
t.Fatalf("X-Stainless-Package-Version = %q, want %q", got, pkgVersion)
}
if got := headers.Get("X-Stainless-Runtime-Version"); got != runtimeVersion {
t.Fatalf("X-Stainless-Runtime-Version = %q, want %q", got, runtimeVersion)
}
if got := headers.Get("X-Stainless-Os"); got != osName {
t.Fatalf("X-Stainless-Os = %q, want %q", got, osName)
}
if got := headers.Get("X-Stainless-Arch"); got != arch {
t.Fatalf("X-Stainless-Arch = %q, want %q", got, arch)
}
}
func TestApplyClaudeHeaders_UsesConfiguredBaselineFingerprint(t *testing.T) {
resetClaudeDeviceProfileCache()
stabilize := true
cfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.70 (external, cli)",
PackageVersion: "0.80.0",
RuntimeVersion: "v24.5.0",
OS: "MacOS",
Arch: "arm64",
Timeout: "900",
StabilizeDeviceProfile: &stabilize,
},
}
auth := &cliproxyauth.Auth{
ID: "auth-baseline",
Attributes: map[string]string{
"api_key": "key-baseline",
"header:User-Agent": "evil-client/9.9",
"header:X-Stainless-Os": "Linux",
"header:X-Stainless-Arch": "x64",
"header:X-Stainless-Package-Version": "9.9.9",
},
}
incoming := http.Header{
"User-Agent": []string{"curl/8.7.1"},
"X-Stainless-Package-Version": []string{"0.10.0"},
"X-Stainless-Runtime-Version": []string{"v18.0.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
}
req := newClaudeHeaderTestRequest(t, incoming)
applyClaudeHeaders(req, auth, "key-baseline", false, nil, cfg)
assertClaudeFingerprint(t, req.Header, "claude-cli/2.1.70 (external, cli)", "0.80.0", "v24.5.0", "MacOS", "arm64")
if got := req.Header.Get("X-Stainless-Timeout"); got != "900" {
t.Fatalf("X-Stainless-Timeout = %q, want %q", got, "900")
}
}
func TestApplyClaudeHeaders_TracksHighestClaudeCLIFingerprint(t *testing.T) {
resetClaudeDeviceProfileCache()
stabilize := true
cfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.60 (external, cli)",
PackageVersion: "0.70.0",
RuntimeVersion: "v22.0.0",
OS: "MacOS",
Arch: "arm64",
StabilizeDeviceProfile: &stabilize,
},
}
auth := &cliproxyauth.Auth{
ID: "auth-upgrade",
Attributes: map[string]string{
"api_key": "key-upgrade",
},
}
firstReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.62 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.74.0"},
"X-Stainless-Runtime-Version": []string{"v24.3.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(firstReq, auth, "key-upgrade", false, nil, cfg)
assertClaudeFingerprint(t, firstReq.Header, "claude-cli/2.1.62 (external, cli)", "0.74.0", "v24.3.0", "MacOS", "arm64")
thirdPartyReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"lobe-chat/1.0"},
"X-Stainless-Package-Version": []string{"0.10.0"},
"X-Stainless-Runtime-Version": []string{"v18.0.0"},
"X-Stainless-Os": []string{"Windows"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(thirdPartyReq, auth, "key-upgrade", false, nil, cfg)
assertClaudeFingerprint(t, thirdPartyReq.Header, "claude-cli/2.1.62 (external, cli)", "0.74.0", "v24.3.0", "MacOS", "arm64")
higherReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.63 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.75.0"},
"X-Stainless-Runtime-Version": []string{"v24.4.0"},
"X-Stainless-Os": []string{"MacOS"},
"X-Stainless-Arch": []string{"arm64"},
})
applyClaudeHeaders(higherReq, auth, "key-upgrade", false, nil, cfg)
assertClaudeFingerprint(t, higherReq.Header, "claude-cli/2.1.63 (external, cli)", "0.75.0", "v24.4.0", "MacOS", "arm64")
lowerReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.61 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.73.0"},
"X-Stainless-Runtime-Version": []string{"v24.2.0"},
"X-Stainless-Os": []string{"Windows"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(lowerReq, auth, "key-upgrade", false, nil, cfg)
assertClaudeFingerprint(t, lowerReq.Header, "claude-cli/2.1.63 (external, cli)", "0.75.0", "v24.4.0", "MacOS", "arm64")
}
func TestApplyClaudeHeaders_DoesNotDowngradeConfiguredBaselineOnFirstClaudeClient(t *testing.T) {
resetClaudeDeviceProfileCache()
stabilize := true
cfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.70 (external, cli)",
PackageVersion: "0.80.0",
RuntimeVersion: "v24.5.0",
OS: "MacOS",
Arch: "arm64",
StabilizeDeviceProfile: &stabilize,
},
}
auth := &cliproxyauth.Auth{
ID: "auth-baseline-floor",
Attributes: map[string]string{
"api_key": "key-baseline-floor",
},
}
olderClaudeReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.62 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.74.0"},
"X-Stainless-Runtime-Version": []string{"v24.3.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(olderClaudeReq, auth, "key-baseline-floor", false, nil, cfg)
assertClaudeFingerprint(t, olderClaudeReq.Header, "claude-cli/2.1.70 (external, cli)", "0.80.0", "v24.5.0", "MacOS", "arm64")
newerClaudeReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.71 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.81.0"},
"X-Stainless-Runtime-Version": []string{"v24.6.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(newerClaudeReq, auth, "key-baseline-floor", false, nil, cfg)
assertClaudeFingerprint(t, newerClaudeReq.Header, "claude-cli/2.1.71 (external, cli)", "0.81.0", "v24.6.0", "MacOS", "arm64")
}
func TestApplyClaudeHeaders_UpgradesCachedSoftwareFingerprintWhenBaselineAdvances(t *testing.T) {
resetClaudeDeviceProfileCache()
stabilize := true
oldCfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.70 (external, cli)",
PackageVersion: "0.80.0",
RuntimeVersion: "v24.5.0",
OS: "MacOS",
Arch: "arm64",
StabilizeDeviceProfile: &stabilize,
},
}
newCfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.77 (external, cli)",
PackageVersion: "0.87.0",
RuntimeVersion: "v24.8.0",
OS: "MacOS",
Arch: "arm64",
StabilizeDeviceProfile: &stabilize,
},
}
auth := &cliproxyauth.Auth{
ID: "auth-baseline-reload",
Attributes: map[string]string{
"api_key": "key-baseline-reload",
},
}
officialReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.71 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.81.0"},
"X-Stainless-Runtime-Version": []string{"v24.6.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(officialReq, auth, "key-baseline-reload", false, nil, oldCfg)
assertClaudeFingerprint(t, officialReq.Header, "claude-cli/2.1.71 (external, cli)", "0.81.0", "v24.6.0", "MacOS", "arm64")
thirdPartyReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"curl/8.7.1"},
"X-Stainless-Package-Version": []string{"0.10.0"},
"X-Stainless-Runtime-Version": []string{"v18.0.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(thirdPartyReq, auth, "key-baseline-reload", false, nil, newCfg)
assertClaudeFingerprint(t, thirdPartyReq.Header, "claude-cli/2.1.77 (external, cli)", "0.87.0", "v24.8.0", "MacOS", "arm64")
}
func TestApplyClaudeHeaders_LearnsOfficialFingerprintAfterCustomBaselineFallback(t *testing.T) {
resetClaudeDeviceProfileCache()
stabilize := true
cfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "my-gateway/1.0",
PackageVersion: "custom-pkg",
RuntimeVersion: "custom-runtime",
OS: "MacOS",
Arch: "arm64",
StabilizeDeviceProfile: &stabilize,
},
}
auth := &cliproxyauth.Auth{
ID: "auth-custom-baseline-learning",
Attributes: map[string]string{
"api_key": "key-custom-baseline-learning",
},
}
thirdPartyReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"curl/8.7.1"},
"X-Stainless-Package-Version": []string{"0.10.0"},
"X-Stainless-Runtime-Version": []string{"v18.0.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(thirdPartyReq, auth, "key-custom-baseline-learning", false, nil, cfg)
assertClaudeFingerprint(t, thirdPartyReq.Header, "my-gateway/1.0", "custom-pkg", "custom-runtime", "MacOS", "arm64")
officialReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.77 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.87.0"},
"X-Stainless-Runtime-Version": []string{"v24.8.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(officialReq, auth, "key-custom-baseline-learning", false, nil, cfg)
assertClaudeFingerprint(t, officialReq.Header, "claude-cli/2.1.77 (external, cli)", "0.87.0", "v24.8.0", "MacOS", "arm64")
postLearningThirdPartyReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"curl/8.7.1"},
"X-Stainless-Package-Version": []string{"0.10.0"},
"X-Stainless-Runtime-Version": []string{"v18.0.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(postLearningThirdPartyReq, auth, "key-custom-baseline-learning", false, nil, cfg)
assertClaudeFingerprint(t, postLearningThirdPartyReq.Header, "claude-cli/2.1.77 (external, cli)", "0.87.0", "v24.8.0", "MacOS", "arm64")
}
func TestResolveClaudeDeviceProfile_RechecksCacheBeforeStoringCandidate(t *testing.T) {
resetClaudeDeviceProfileCache()
stabilize := true
cfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.60 (external, cli)",
PackageVersion: "0.70.0",
RuntimeVersion: "v22.0.0",
OS: "MacOS",
Arch: "arm64",
StabilizeDeviceProfile: &stabilize,
},
}
auth := &cliproxyauth.Auth{
ID: "auth-racy-upgrade",
Attributes: map[string]string{
"api_key": "key-racy-upgrade",
},
}
lowPaused := make(chan struct{})
releaseLow := make(chan struct{})
var pauseOnce sync.Once
var releaseOnce sync.Once
claudeDeviceProfileBeforeCandidateStore = func(candidate claudeDeviceProfile) {
if candidate.UserAgent != "claude-cli/2.1.62 (external, cli)" {
return
}
pauseOnce.Do(func() { close(lowPaused) })
<-releaseLow
}
t.Cleanup(func() {
claudeDeviceProfileBeforeCandidateStore = nil
releaseOnce.Do(func() { close(releaseLow) })
})
lowResultCh := make(chan claudeDeviceProfile, 1)
go func() {
lowResultCh <- resolveClaudeDeviceProfile(auth, "key-racy-upgrade", http.Header{
"User-Agent": []string{"claude-cli/2.1.62 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.74.0"},
"X-Stainless-Runtime-Version": []string{"v24.3.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
}, cfg)
}()
select {
case <-lowPaused:
case <-time.After(2 * time.Second):
t.Fatal("timed out waiting for lower candidate to pause before storing")
}
highResult := resolveClaudeDeviceProfile(auth, "key-racy-upgrade", http.Header{
"User-Agent": []string{"claude-cli/2.1.63 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.75.0"},
"X-Stainless-Runtime-Version": []string{"v24.4.0"},
"X-Stainless-Os": []string{"MacOS"},
"X-Stainless-Arch": []string{"arm64"},
}, cfg)
releaseOnce.Do(func() { close(releaseLow) })
select {
case lowResult := <-lowResultCh:
if lowResult.UserAgent != "claude-cli/2.1.63 (external, cli)" {
t.Fatalf("lowResult.UserAgent = %q, want %q", lowResult.UserAgent, "claude-cli/2.1.63 (external, cli)")
}
if lowResult.PackageVersion != "0.75.0" {
t.Fatalf("lowResult.PackageVersion = %q, want %q", lowResult.PackageVersion, "0.75.0")
}
if lowResult.OS != "MacOS" || lowResult.Arch != "arm64" {
t.Fatalf("lowResult platform = %s/%s, want %s/%s", lowResult.OS, lowResult.Arch, "MacOS", "arm64")
}
case <-time.After(2 * time.Second):
t.Fatal("timed out waiting for lower candidate result")
}
if highResult.UserAgent != "claude-cli/2.1.63 (external, cli)" {
t.Fatalf("highResult.UserAgent = %q, want %q", highResult.UserAgent, "claude-cli/2.1.63 (external, cli)")
}
if highResult.OS != "MacOS" || highResult.Arch != "arm64" {
t.Fatalf("highResult platform = %s/%s, want %s/%s", highResult.OS, highResult.Arch, "MacOS", "arm64")
}
cached := resolveClaudeDeviceProfile(auth, "key-racy-upgrade", http.Header{
"User-Agent": []string{"curl/8.7.1"},
}, cfg)
if cached.UserAgent != "claude-cli/2.1.63 (external, cli)" {
t.Fatalf("cached.UserAgent = %q, want %q", cached.UserAgent, "claude-cli/2.1.63 (external, cli)")
}
if cached.PackageVersion != "0.75.0" {
t.Fatalf("cached.PackageVersion = %q, want %q", cached.PackageVersion, "0.75.0")
}
if cached.OS != "MacOS" || cached.Arch != "arm64" {
t.Fatalf("cached platform = %s/%s, want %s/%s", cached.OS, cached.Arch, "MacOS", "arm64")
}
}
func TestApplyClaudeHeaders_ThirdPartyBaselineThenOfficialUpgradeKeepsPinnedPlatform(t *testing.T) {
resetClaudeDeviceProfileCache()
stabilize := true
cfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.70 (external, cli)",
PackageVersion: "0.80.0",
RuntimeVersion: "v24.5.0",
OS: "MacOS",
Arch: "arm64",
StabilizeDeviceProfile: &stabilize,
},
}
auth := &cliproxyauth.Auth{
ID: "auth-third-party-then-official",
Attributes: map[string]string{
"api_key": "key-third-party-then-official",
},
}
thirdPartyReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"curl/8.7.1"},
"X-Stainless-Package-Version": []string{"0.10.0"},
"X-Stainless-Runtime-Version": []string{"v18.0.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(thirdPartyReq, auth, "key-third-party-then-official", false, nil, cfg)
assertClaudeFingerprint(t, thirdPartyReq.Header, "claude-cli/2.1.70 (external, cli)", "0.80.0", "v24.5.0", "MacOS", "arm64")
officialReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.77 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.87.0"},
"X-Stainless-Runtime-Version": []string{"v24.8.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(officialReq, auth, "key-third-party-then-official", false, nil, cfg)
assertClaudeFingerprint(t, officialReq.Header, "claude-cli/2.1.77 (external, cli)", "0.87.0", "v24.8.0", "MacOS", "arm64")
}
func TestApplyClaudeHeaders_DisableDeviceProfileStabilization(t *testing.T) {
resetClaudeDeviceProfileCache()
stabilize := false
cfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.60 (external, cli)",
PackageVersion: "0.70.0",
RuntimeVersion: "v22.0.0",
OS: "MacOS",
Arch: "arm64",
StabilizeDeviceProfile: &stabilize,
},
}
auth := &cliproxyauth.Auth{
ID: "auth-disable-stability",
Attributes: map[string]string{
"api_key": "key-disable-stability",
},
}
firstReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.62 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.74.0"},
"X-Stainless-Runtime-Version": []string{"v24.3.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(firstReq, auth, "key-disable-stability", false, nil, cfg)
assertClaudeFingerprint(t, firstReq.Header, "claude-cli/2.1.62 (external, cli)", "0.74.0", "v24.3.0", "Linux", "x64")
thirdPartyReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"lobe-chat/1.0"},
"X-Stainless-Package-Version": []string{"0.10.0"},
"X-Stainless-Runtime-Version": []string{"v18.0.0"},
"X-Stainless-Os": []string{"Windows"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(thirdPartyReq, auth, "key-disable-stability", false, nil, cfg)
assertClaudeFingerprint(t, thirdPartyReq.Header, "claude-cli/2.1.60 (external, cli)", "0.10.0", "v18.0.0", "Windows", "x64")
lowerReq := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.61 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.73.0"},
"X-Stainless-Runtime-Version": []string{"v24.2.0"},
"X-Stainless-Os": []string{"Windows"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(lowerReq, auth, "key-disable-stability", false, nil, cfg)
assertClaudeFingerprint(t, lowerReq.Header, "claude-cli/2.1.61 (external, cli)", "0.73.0", "v24.2.0", "Windows", "x64")
}
func TestApplyClaudeHeaders_LegacyModePreservesConfiguredUserAgentOverrideForClaudeClients(t *testing.T) {
resetClaudeDeviceProfileCache()
stabilize := false
cfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.60 (external, cli)",
PackageVersion: "0.70.0",
RuntimeVersion: "v22.0.0",
StabilizeDeviceProfile: &stabilize,
},
}
auth := &cliproxyauth.Auth{
ID: "auth-legacy-ua-override",
Attributes: map[string]string{
"api_key": "key-legacy-ua-override",
"header:User-Agent": "config-ua/1.0",
},
}
req := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"claude-cli/2.1.62 (external, cli)"},
"X-Stainless-Package-Version": []string{"0.74.0"},
"X-Stainless-Runtime-Version": []string{"v24.3.0"},
"X-Stainless-Os": []string{"Linux"},
"X-Stainless-Arch": []string{"x64"},
})
applyClaudeHeaders(req, auth, "key-legacy-ua-override", false, nil, cfg)
assertClaudeFingerprint(t, req.Header, "config-ua/1.0", "0.74.0", "v24.3.0", "Linux", "x64")
}
func TestApplyClaudeHeaders_LegacyModeFallsBackToRuntimeOSArchWhenMissing(t *testing.T) {
resetClaudeDeviceProfileCache()
stabilize := false
cfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.60 (external, cli)",
PackageVersion: "0.70.0",
RuntimeVersion: "v22.0.0",
OS: "MacOS",
Arch: "arm64",
StabilizeDeviceProfile: &stabilize,
},
}
auth := &cliproxyauth.Auth{
ID: "auth-legacy-runtime-os-arch",
Attributes: map[string]string{
"api_key": "key-legacy-runtime-os-arch",
},
}
req := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"curl/8.7.1"},
})
applyClaudeHeaders(req, auth, "key-legacy-runtime-os-arch", false, nil, cfg)
assertClaudeFingerprint(t, req.Header, "claude-cli/2.1.60 (external, cli)", "0.70.0", "v22.0.0", mapStainlessOS(), mapStainlessArch())
}
func TestApplyClaudeHeaders_UnsetStabilizationAlsoUsesLegacyRuntimeOSArchFallback(t *testing.T) {
resetClaudeDeviceProfileCache()
cfg := &config.Config{
ClaudeHeaderDefaults: config.ClaudeHeaderDefaults{
UserAgent: "claude-cli/2.1.60 (external, cli)",
PackageVersion: "0.70.0",
RuntimeVersion: "v22.0.0",
OS: "MacOS",
Arch: "arm64",
},
}
auth := &cliproxyauth.Auth{
ID: "auth-unset-runtime-os-arch",
Attributes: map[string]string{
"api_key": "key-unset-runtime-os-arch",
},
}
req := newClaudeHeaderTestRequest(t, http.Header{
"User-Agent": []string{"curl/8.7.1"},
})
applyClaudeHeaders(req, auth, "key-unset-runtime-os-arch", false, nil, cfg)
assertClaudeFingerprint(t, req.Header, "claude-cli/2.1.60 (external, cli)", "0.70.0", "v22.0.0", mapStainlessOS(), mapStainlessArch())
}
func TestClaudeDeviceProfileStabilizationEnabled_DefaultFalse(t *testing.T) {
if claudeDeviceProfileStabilizationEnabled(nil) {
t.Fatal("expected nil config to default to disabled stabilization")
}
if claudeDeviceProfileStabilizationEnabled(&config.Config{}) {
t.Fatal("expected unset stabilize-device-profile to default to disabled stabilization")
}
}
func TestApplyClaudeToolPrefix(t *testing.T) {
input := []byte(`{"tools":[{"name":"alpha"},{"name":"proxy_bravo"}],"tool_choice":{"type":"tool","name":"charlie"},"messages":[{"role":"assistant","content":[{"type":"tool_use","name":"delta","id":"t1","input":{}}]}]}`)
out := applyClaudeToolPrefix(input, "proxy_")
@@ -599,6 +1184,83 @@ func testClaudeExecutorInvalidCompressedErrorBody(
}
}
func TestEnsureModelMaxTokens_UsesRegisteredMaxCompletionTokens(t *testing.T) {
reg := registry.GetGlobalRegistry()
clientID := "test-claude-max-completion-tokens-client"
modelID := "test-claude-max-completion-tokens-model"
reg.RegisterClient(clientID, "claude", []*registry.ModelInfo{{
ID: modelID,
Type: "claude",
OwnedBy: "anthropic",
Object: "model",
Created: time.Now().Unix(),
MaxCompletionTokens: 4096,
UserDefined: true,
}})
defer reg.UnregisterClient(clientID)
input := []byte(`{"model":"test-claude-max-completion-tokens-model","messages":[{"role":"user","content":"hi"}]}`)
out := ensureModelMaxTokens(input, modelID)
if got := gjson.GetBytes(out, "max_tokens").Int(); got != 4096 {
t.Fatalf("max_tokens = %d, want %d", got, 4096)
}
}
func TestEnsureModelMaxTokens_DefaultsMissingValue(t *testing.T) {
reg := registry.GetGlobalRegistry()
clientID := "test-claude-default-max-tokens-client"
modelID := "test-claude-default-max-tokens-model"
reg.RegisterClient(clientID, "claude", []*registry.ModelInfo{{
ID: modelID,
Type: "claude",
OwnedBy: "anthropic",
Object: "model",
Created: time.Now().Unix(),
UserDefined: true,
}})
defer reg.UnregisterClient(clientID)
input := []byte(`{"model":"test-claude-default-max-tokens-model","messages":[{"role":"user","content":"hi"}]}`)
out := ensureModelMaxTokens(input, modelID)
if got := gjson.GetBytes(out, "max_tokens").Int(); got != defaultModelMaxTokens {
t.Fatalf("max_tokens = %d, want %d", got, defaultModelMaxTokens)
}
}
func TestEnsureModelMaxTokens_PreservesExplicitValue(t *testing.T) {
reg := registry.GetGlobalRegistry()
clientID := "test-claude-preserve-max-tokens-client"
modelID := "test-claude-preserve-max-tokens-model"
reg.RegisterClient(clientID, "claude", []*registry.ModelInfo{{
ID: modelID,
Type: "claude",
OwnedBy: "anthropic",
Object: "model",
Created: time.Now().Unix(),
MaxCompletionTokens: 4096,
UserDefined: true,
}})
defer reg.UnregisterClient(clientID)
input := []byte(`{"model":"test-claude-preserve-max-tokens-model","max_tokens":2048,"messages":[{"role":"user","content":"hi"}]}`)
out := ensureModelMaxTokens(input, modelID)
if got := gjson.GetBytes(out, "max_tokens").Int(); got != 2048 {
t.Fatalf("max_tokens = %d, want %d", got, 2048)
}
}
func TestEnsureModelMaxTokens_SkipsUnregisteredModel(t *testing.T) {
input := []byte(`{"model":"test-claude-unregistered-model","messages":[{"role":"user","content":"hi"}]}`)
out := ensureModelMaxTokens(input, "test-claude-unregistered-model")
if gjson.GetBytes(out, "max_tokens").Exists() {
t.Fatalf("max_tokens should remain unset, got %s", gjson.GetBytes(out, "max_tokens").Raw)
}
}
// TestClaudeExecutor_ExecuteStream_SetsIdentityAcceptEncoding verifies that streaming
// requests use Accept-Encoding: identity so the upstream cannot respond with a
// compressed SSE body that would silently break the line scanner.
+41 -6
View File
@@ -28,8 +28,8 @@ import (
)
const (
codexClientVersion = "0.101.0"
codexUserAgent = "codex_cli_rs/0.101.0 (Mac OS 26.0.1; arm64) Apple_Terminal/464"
codexUserAgent = "codex_cli_rs/0.116.0 (Mac OS 26.0.1; arm64) Apple_Terminal/464"
codexOriginator = "codex_cli_rs"
)
var dataTag = []byte("data:")
@@ -113,6 +113,7 @@ func (e *CodexExecutor) Execute(ctx context.Context, auth *cliproxyauth.Auth, re
body, _ = sjson.DeleteBytes(body, "previous_response_id")
body, _ = sjson.DeleteBytes(body, "prompt_cache_retention")
body, _ = sjson.DeleteBytes(body, "safety_identifier")
body, _ = sjson.DeleteBytes(body, "stream_options")
if !gjson.GetBytes(body, "instructions").Exists() {
body, _ = sjson.SetBytes(body, "instructions", "")
}
@@ -311,6 +312,7 @@ func (e *CodexExecutor) ExecuteStream(ctx context.Context, auth *cliproxyauth.Au
body, _ = sjson.DeleteBytes(body, "previous_response_id")
body, _ = sjson.DeleteBytes(body, "prompt_cache_retention")
body, _ = sjson.DeleteBytes(body, "safety_identifier")
body, _ = sjson.DeleteBytes(body, "stream_options")
body, _ = sjson.SetBytes(body, "model", baseModel)
if !gjson.GetBytes(body, "instructions").Exists() {
body, _ = sjson.SetBytes(body, "instructions", "")
@@ -415,6 +417,7 @@ func (e *CodexExecutor) CountTokens(ctx context.Context, auth *cliproxyauth.Auth
body, _ = sjson.DeleteBytes(body, "previous_response_id")
body, _ = sjson.DeleteBytes(body, "prompt_cache_retention")
body, _ = sjson.DeleteBytes(body, "safety_identifier")
body, _ = sjson.DeleteBytes(body, "stream_options")
body, _ = sjson.SetBytes(body, "stream", false)
if !gjson.GetBytes(body, "instructions").Exists() {
body, _ = sjson.SetBytes(body, "instructions", "")
@@ -645,8 +648,10 @@ func applyCodexHeaders(r *http.Request, auth *cliproxyauth.Auth, token string, s
ginHeaders = ginCtx.Request.Header
}
misc.EnsureHeader(r.Header, ginHeaders, "Version", codexClientVersion)
misc.EnsureHeader(r.Header, ginHeaders, "Version", "")
misc.EnsureHeader(r.Header, ginHeaders, "Session_id", uuid.NewString())
misc.EnsureHeader(r.Header, ginHeaders, "X-Codex-Turn-Metadata", "")
misc.EnsureHeader(r.Header, ginHeaders, "X-Client-Request-Id", "")
cfgUserAgent, _ := codexHeaderDefaults(cfg, auth)
ensureHeaderWithConfigPrecedence(r.Header, ginHeaders, "User-Agent", cfgUserAgent, codexUserAgent)
@@ -663,8 +668,12 @@ func applyCodexHeaders(r *http.Request, auth *cliproxyauth.Auth, token string, s
isAPIKey = true
}
}
if originator := strings.TrimSpace(ginHeaders.Get("Originator")); originator != "" {
r.Header.Set("Originator", originator)
} else if !isAPIKey {
r.Header.Set("Originator", codexOriginator)
}
if !isAPIKey {
r.Header.Set("Originator", "codex_cli_rs")
if auth != nil && auth.Metadata != nil {
if accountID, ok := auth.Metadata["account_id"].(string); ok {
r.Header.Set("Chatgpt-Account-Id", accountID)
@@ -679,13 +688,39 @@ func applyCodexHeaders(r *http.Request, auth *cliproxyauth.Auth, token string, s
}
func newCodexStatusErr(statusCode int, body []byte) statusErr {
err := statusErr{code: statusCode, msg: string(body)}
if retryAfter := parseCodexRetryAfter(statusCode, body, time.Now()); retryAfter != nil {
errCode := statusCode
if isCodexModelCapacityError(body) {
errCode = http.StatusTooManyRequests
}
err := statusErr{code: errCode, msg: string(body)}
if retryAfter := parseCodexRetryAfter(errCode, body, time.Now()); retryAfter != nil {
err.retryAfter = retryAfter
}
return err
}
func isCodexModelCapacityError(errorBody []byte) bool {
if len(errorBody) == 0 {
return false
}
candidates := []string{
gjson.GetBytes(errorBody, "error.message").String(),
gjson.GetBytes(errorBody, "message").String(),
string(errorBody),
}
for _, candidate := range candidates {
lower := strings.ToLower(strings.TrimSpace(candidate))
if lower == "" {
continue
}
if strings.Contains(lower, "selected model is at capacity") ||
strings.Contains(lower, "model is at capacity. please try a different model") {
return true
}
}
return false
}
func parseCodexRetryAfter(statusCode int, errorBody []byte, now time.Time) *time.Duration {
if statusCode != http.StatusTooManyRequests || len(errorBody) == 0 {
return nil
@@ -60,6 +60,19 @@ func TestParseCodexRetryAfter(t *testing.T) {
})
}
func TestNewCodexStatusErrTreatsCapacityAsRetryableRateLimit(t *testing.T) {
body := []byte(`{"error":{"message":"Selected model is at capacity. Please try a different model."}}`)
err := newCodexStatusErr(http.StatusBadRequest, body)
if got := err.StatusCode(); got != http.StatusTooManyRequests {
t.Fatalf("status code = %d, want %d", got, http.StatusTooManyRequests)
}
if err.RetryAfter() != nil {
t.Fatalf("expected nil explicit retryAfter for capacity fallback, got %v", *err.RetryAfter())
}
}
func itoa(v int64) string {
return strconv.FormatInt(v, 10)
}
@@ -814,9 +814,10 @@ func applyCodexWebsocketHeaders(ctx context.Context, headers http.Header, auth *
ensureHeaderWithPriority(headers, ginHeaders, "x-codex-beta-features", cfgBetaFeatures, "")
misc.EnsureHeader(headers, ginHeaders, "x-codex-turn-state", "")
misc.EnsureHeader(headers, ginHeaders, "x-codex-turn-metadata", "")
misc.EnsureHeader(headers, ginHeaders, "x-client-request-id", "")
misc.EnsureHeader(headers, ginHeaders, "x-responsesapi-include-timing-metrics", "")
misc.EnsureHeader(headers, ginHeaders, "Version", "")
misc.EnsureHeader(headers, ginHeaders, "Version", codexClientVersion)
betaHeader := strings.TrimSpace(headers.Get("OpenAI-Beta"))
if betaHeader == "" && ginHeaders != nil {
betaHeader = strings.TrimSpace(ginHeaders.Get("OpenAI-Beta"))
@@ -834,8 +835,12 @@ func applyCodexWebsocketHeaders(ctx context.Context, headers http.Header, auth *
isAPIKey = true
}
}
if originator := strings.TrimSpace(ginHeaders.Get("Originator")); originator != "" {
headers.Set("Originator", originator)
} else if !isAPIKey {
headers.Set("Originator", codexOriginator)
}
if !isAPIKey {
headers.Set("Originator", "codex_cli_rs")
if auth != nil && auth.Metadata != nil {
if accountID, ok := auth.Metadata["account_id"].(string); ok {
if trimmed := strings.TrimSpace(accountID); trimmed != "" {
@@ -41,9 +41,46 @@ func TestApplyCodexWebsocketHeadersDefaultsToCurrentResponsesBeta(t *testing.T)
if got := headers.Get("User-Agent"); got != codexUserAgent {
t.Fatalf("User-Agent = %s, want %s", got, codexUserAgent)
}
if got := headers.Get("Version"); got != "" {
t.Fatalf("Version = %q, want empty", got)
}
if got := headers.Get("x-codex-beta-features"); got != "" {
t.Fatalf("x-codex-beta-features = %q, want empty", got)
}
if got := headers.Get("X-Codex-Turn-Metadata"); got != "" {
t.Fatalf("X-Codex-Turn-Metadata = %q, want empty", got)
}
if got := headers.Get("X-Client-Request-Id"); got != "" {
t.Fatalf("X-Client-Request-Id = %q, want empty", got)
}
}
func TestApplyCodexWebsocketHeadersPassesThroughClientIdentityHeaders(t *testing.T) {
auth := &cliproxyauth.Auth{
Provider: "codex",
Metadata: map[string]any{"email": "user@example.com"},
}
ctx := contextWithGinHeaders(map[string]string{
"Originator": "Codex Desktop",
"Version": "0.115.0-alpha.27",
"X-Codex-Turn-Metadata": `{"turn_id":"turn-1"}`,
"X-Client-Request-Id": "019d2233-e240-7162-992d-38df0a2a0e0d",
})
headers := applyCodexWebsocketHeaders(ctx, http.Header{}, auth, "", nil)
if got := headers.Get("Originator"); got != "Codex Desktop" {
t.Fatalf("Originator = %s, want %s", got, "Codex Desktop")
}
if got := headers.Get("Version"); got != "0.115.0-alpha.27" {
t.Fatalf("Version = %s, want %s", got, "0.115.0-alpha.27")
}
if got := headers.Get("X-Codex-Turn-Metadata"); got != `{"turn_id":"turn-1"}` {
t.Fatalf("X-Codex-Turn-Metadata = %s, want %s", got, `{"turn_id":"turn-1"}`)
}
if got := headers.Get("X-Client-Request-Id"); got != "019d2233-e240-7162-992d-38df0a2a0e0d" {
t.Fatalf("X-Client-Request-Id = %s, want %s", got, "019d2233-e240-7162-992d-38df0a2a0e0d")
}
}
func TestApplyCodexWebsocketHeadersUsesConfigDefaultsForOAuth(t *testing.T) {
@@ -177,6 +214,57 @@ func TestApplyCodexHeadersUsesConfigUserAgentForOAuth(t *testing.T) {
}
}
func TestApplyCodexHeadersPassesThroughClientIdentityHeaders(t *testing.T) {
req, err := http.NewRequest(http.MethodPost, "https://example.com/responses", nil)
if err != nil {
t.Fatalf("NewRequest() error = %v", err)
}
auth := &cliproxyauth.Auth{
Provider: "codex",
Metadata: map[string]any{"email": "user@example.com"},
}
req = req.WithContext(contextWithGinHeaders(map[string]string{
"Originator": "Codex Desktop",
"Version": "0.115.0-alpha.27",
"X-Codex-Turn-Metadata": `{"turn_id":"turn-1"}`,
"X-Client-Request-Id": "019d2233-e240-7162-992d-38df0a2a0e0d",
}))
applyCodexHeaders(req, auth, "oauth-token", true, nil)
if got := req.Header.Get("Originator"); got != "Codex Desktop" {
t.Fatalf("Originator = %s, want %s", got, "Codex Desktop")
}
if got := req.Header.Get("Version"); got != "0.115.0-alpha.27" {
t.Fatalf("Version = %s, want %s", got, "0.115.0-alpha.27")
}
if got := req.Header.Get("X-Codex-Turn-Metadata"); got != `{"turn_id":"turn-1"}` {
t.Fatalf("X-Codex-Turn-Metadata = %s, want %s", got, `{"turn_id":"turn-1"}`)
}
if got := req.Header.Get("X-Client-Request-Id"); got != "019d2233-e240-7162-992d-38df0a2a0e0d" {
t.Fatalf("X-Client-Request-Id = %s, want %s", got, "019d2233-e240-7162-992d-38df0a2a0e0d")
}
}
func TestApplyCodexHeadersDoesNotInjectClientOnlyHeadersByDefault(t *testing.T) {
req, err := http.NewRequest(http.MethodPost, "https://example.com/responses", nil)
if err != nil {
t.Fatalf("NewRequest() error = %v", err)
}
applyCodexHeaders(req, nil, "oauth-token", true, nil)
if got := req.Header.Get("Version"); got != "" {
t.Fatalf("Version = %q, want empty", got)
}
if got := req.Header.Get("X-Codex-Turn-Metadata"); got != "" {
t.Fatalf("X-Codex-Turn-Metadata = %q, want empty", got)
}
if got := req.Header.Get("X-Client-Request-Id"); got != "" {
t.Fatalf("X-Client-Request-Id = %q, want empty", got)
}
}
func contextWithGinHeaders(headers map[string]string) context.Context {
gin.SetMode(gin.TestMode)
recorder := httptest.NewRecorder()
+31 -22
View File
@@ -73,17 +73,7 @@ func (r *usageReporter) publishWithOutcome(ctx context.Context, detail usage.Det
return
}
r.once.Do(func() {
usage.PublishRecord(ctx, usage.Record{
Provider: r.provider,
Model: r.model,
Source: r.source,
APIKey: r.apiKey,
AuthID: r.authID,
AuthIndex: r.authIndex,
RequestedAt: r.requestedAt,
Failed: failed,
Detail: detail,
})
usage.PublishRecord(ctx, r.buildRecord(detail, failed))
})
}
@@ -96,20 +86,39 @@ func (r *usageReporter) ensurePublished(ctx context.Context) {
return
}
r.once.Do(func() {
usage.PublishRecord(ctx, usage.Record{
Provider: r.provider,
Model: r.model,
Source: r.source,
APIKey: r.apiKey,
AuthID: r.authID,
AuthIndex: r.authIndex,
RequestedAt: r.requestedAt,
Failed: false,
Detail: usage.Detail{},
})
usage.PublishRecord(ctx, r.buildRecord(usage.Detail{}, false))
})
}
func (r *usageReporter) buildRecord(detail usage.Detail, failed bool) usage.Record {
if r == nil {
return usage.Record{Detail: detail, Failed: failed}
}
return usage.Record{
Provider: r.provider,
Model: r.model,
Source: r.source,
APIKey: r.apiKey,
AuthID: r.authID,
AuthIndex: r.authIndex,
RequestedAt: r.requestedAt,
Latency: r.latency(),
Failed: failed,
Detail: detail,
}
}
func (r *usageReporter) latency() time.Duration {
if r == nil || r.requestedAt.IsZero() {
return 0
}
latency := time.Since(r.requestedAt)
if latency < 0 {
return 0
}
return latency
}
func apiKeyFromContext(ctx context.Context) string {
if ctx == nil {
return ""
@@ -1,6 +1,11 @@
package executor
import "testing"
import (
"testing"
"time"
"github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/usage"
)
func TestParseOpenAIUsageChatCompletions(t *testing.T) {
data := []byte(`{"usage":{"prompt_tokens":1,"completion_tokens":2,"total_tokens":3,"prompt_tokens_details":{"cached_tokens":4},"completion_tokens_details":{"reasoning_tokens":5}}}`)
@@ -41,3 +46,19 @@ func TestParseOpenAIUsageResponses(t *testing.T) {
t.Fatalf("reasoning tokens = %d, want %d", detail.ReasoningTokens, 9)
}
}
func TestUsageReporterBuildRecordIncludesLatency(t *testing.T) {
reporter := &usageReporter{
provider: "openai",
model: "gpt-5.4",
requestedAt: time.Now().Add(-1500 * time.Millisecond),
}
record := reporter.buildRecord(usage.Detail{TotalTokens: 3}, false)
if record.Latency < time.Second {
t.Fatalf("latency = %v, want >= 1s", record.Latency)
}
if record.Latency > 3*time.Second {
t.Fatalf("latency = %v, want <= 3s", record.Latency)
}
}
@@ -148,11 +148,11 @@ func ConvertClaudeRequestToAntigravity(modelName string, inputRawJSON []byte, _
}
// Valid signature, send as thought block
// Always include "text" field — Google Antigravity API requires it
// even for redacted thinking where the text is empty.
partJSON := []byte(`{}`)
partJSON, _ = sjson.SetBytes(partJSON, "thought", true)
if thinkingText != "" {
partJSON, _ = sjson.SetBytes(partJSON, "text", thinkingText)
}
partJSON, _ = sjson.SetBytes(partJSON, "text", thinkingText)
if signature != "" {
partJSON, _ = sjson.SetBytes(partJSON, "thoughtSignature", signature)
}
@@ -171,7 +171,7 @@ func ConvertClaudeRequestToAntigravity(modelName string, inputRawJSON []byte, _
// NOTE: Do NOT inject dummy thinking blocks here.
// Antigravity API validates signatures, so dummy values are rejected.
functionName := contentResult.Get("name").String()
functionName := util.SanitizeFunctionName(contentResult.Get("name").String())
argsResult := contentResult.Get("input")
functionID := contentResult.Get("id").String()
@@ -233,7 +233,7 @@ func ConvertClaudeRequestToAntigravity(modelName string, inputRawJSON []byte, _
functionResponseJSON := []byte(`{}`)
functionResponseJSON, _ = sjson.SetBytes(functionResponseJSON, "id", toolCallID)
functionResponseJSON, _ = sjson.SetBytes(functionResponseJSON, "name", funcName)
functionResponseJSON, _ = sjson.SetBytes(functionResponseJSON, "name", util.SanitizeFunctionName(funcName))
responseData := ""
if functionResponseResult.Type == gjson.String {
@@ -330,32 +330,45 @@ func ConvertClaudeRequestToAntigravity(modelName string, inputRawJSON []byte, _
}
}
// Reorder parts for 'model' role to ensure thinking block is first
// Reorder parts for 'model' role:
// 1. Thinking parts first (Antigravity API requirement)
// 2. Regular parts (text, inlineData, etc.)
// 3. FunctionCall parts last
//
// Moving functionCall parts to the end prevents tool_use↔tool_result
// pairing breakage: the Antigravity API internally splits model messages
// at functionCall boundaries. If a text part follows a functionCall, the
// split creates an extra assistant turn between tool_use and tool_result,
// which Claude rejects with "tool_use ids were found without tool_result
// blocks immediately after".
if role == "model" {
partsResult := gjson.GetBytes(clientContentJSON, "parts")
if partsResult.IsArray() {
parts := partsResult.Array()
var thinkingParts []gjson.Result
var otherParts []gjson.Result
for _, part := range parts {
if part.Get("thought").Bool() {
thinkingParts = append(thinkingParts, part)
} else {
otherParts = append(otherParts, part)
}
}
if len(thinkingParts) > 0 {
firstPartIsThinking := parts[0].Get("thought").Bool()
if !firstPartIsThinking || len(thinkingParts) > 1 {
var newParts []interface{}
for _, p := range thinkingParts {
newParts = append(newParts, p.Value())
if len(parts) > 1 {
var thinkingParts []gjson.Result
var regularParts []gjson.Result
var functionCallParts []gjson.Result
for _, part := range parts {
if part.Get("thought").Bool() {
thinkingParts = append(thinkingParts, part)
} else if part.Get("functionCall").Exists() {
functionCallParts = append(functionCallParts, part)
} else {
regularParts = append(regularParts, part)
}
for _, p := range otherParts {
newParts = append(newParts, p.Value())
}
clientContentJSON, _ = sjson.SetBytes(clientContentJSON, "parts", newParts)
}
var newParts []interface{}
for _, p := range thinkingParts {
newParts = append(newParts, p.Value())
}
for _, p := range regularParts {
newParts = append(newParts, p.Value())
}
for _, p := range functionCallParts {
newParts = append(newParts, p.Value())
}
clientContentJSON, _ = sjson.SetBytes(clientContentJSON, "parts", newParts)
}
}
}
@@ -398,6 +411,7 @@ func ConvertClaudeRequestToAntigravity(modelName string, inputRawJSON []byte, _
inputSchema := util.CleanJSONSchemaForAntigravity(inputSchemaResult.Raw)
tool, _ := sjson.DeleteBytes([]byte(toolResult.Raw), "input_schema")
tool, _ = sjson.SetRawBytes(tool, "parametersJsonSchema", []byte(inputSchema))
tool, _ = sjson.SetBytes(tool, "name", util.SanitizeFunctionName(gjson.GetBytes(tool, "name").String()))
for toolKey := range gjson.ParseBytes(tool).Map() {
if util.InArray(allowedToolKeys, toolKey) {
continue
@@ -471,7 +485,7 @@ func ConvertClaudeRequestToAntigravity(modelName string, inputRawJSON []byte, _
case "tool":
out, _ = sjson.SetBytes(out, "request.toolConfig.functionCallingConfig.mode", "ANY")
if toolChoiceName != "" {
out, _ = sjson.SetBytes(out, "request.toolConfig.functionCallingConfig.allowedFunctionNames", []string{toolChoiceName})
out, _ = sjson.SetBytes(out, "request.toolConfig.functionCallingConfig.allowedFunctionNames", []string{util.SanitizeFunctionName(toolChoiceName)})
}
}
}
@@ -361,6 +361,167 @@ func TestConvertClaudeRequestToAntigravity_ReorderThinking(t *testing.T) {
}
}
func TestConvertClaudeRequestToAntigravity_ReorderTextAfterFunctionCall(t *testing.T) {
// Bug: text part after tool_use in an assistant message causes Antigravity
// to split at functionCall boundary, creating an extra assistant turn that
// breaks tool_use↔tool_result adjacency (upstream issue #989).
// Fix: reorder parts so functionCall comes last.
inputJSON := []byte(`{
"model": "claude-sonnet-4-5",
"messages": [
{
"role": "assistant",
"content": [
{"type": "text", "text": "Let me check..."},
{
"type": "tool_use",
"id": "call_abc",
"name": "Read",
"input": {"file": "test.go"}
},
{"type": "text", "text": "Reading the file now"}
]
},
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "call_abc",
"content": "file content"
}
]
}
]
}`)
output := ConvertClaudeRequestToAntigravity("claude-sonnet-4-5", inputJSON, false)
outputStr := string(output)
parts := gjson.Get(outputStr, "request.contents.0.parts").Array()
if len(parts) != 3 {
t.Fatalf("Expected 3 parts, got %d", len(parts))
}
// Text parts should come before functionCall
if parts[0].Get("text").String() != "Let me check..." {
t.Errorf("Expected first text part first, got %s", parts[0].Raw)
}
if parts[1].Get("text").String() != "Reading the file now" {
t.Errorf("Expected second text part second, got %s", parts[1].Raw)
}
if !parts[2].Get("functionCall").Exists() {
t.Errorf("Expected functionCall last, got %s", parts[2].Raw)
}
if parts[2].Get("functionCall.name").String() != "Read" {
t.Errorf("Expected functionCall name 'Read', got '%s'", parts[2].Get("functionCall.name").String())
}
}
func TestConvertClaudeRequestToAntigravity_ReorderParallelFunctionCalls(t *testing.T) {
inputJSON := []byte(`{
"model": "claude-sonnet-4-5",
"messages": [
{
"role": "assistant",
"content": [
{"type": "text", "text": "Reading both files."},
{
"type": "tool_use",
"id": "call_1",
"name": "Read",
"input": {"file": "a.go"}
},
{"type": "text", "text": "And this one too."},
{
"type": "tool_use",
"id": "call_2",
"name": "Read",
"input": {"file": "b.go"}
}
]
}
]
}`)
output := ConvertClaudeRequestToAntigravity("claude-sonnet-4-5", inputJSON, false)
outputStr := string(output)
parts := gjson.Get(outputStr, "request.contents.0.parts").Array()
if len(parts) != 4 {
t.Fatalf("Expected 4 parts, got %d", len(parts))
}
if parts[0].Get("text").String() != "Reading both files." {
t.Errorf("Expected first text, got %s", parts[0].Raw)
}
if parts[1].Get("text").String() != "And this one too." {
t.Errorf("Expected second text, got %s", parts[1].Raw)
}
if parts[2].Get("functionCall.name").String() != "Read" || parts[2].Get("functionCall.id").String() != "call_1" {
t.Errorf("Expected fc1 third, got %s", parts[2].Raw)
}
if parts[3].Get("functionCall.name").String() != "Read" || parts[3].Get("functionCall.id").String() != "call_2" {
t.Errorf("Expected fc2 fourth, got %s", parts[3].Raw)
}
}
func TestConvertClaudeRequestToAntigravity_ReorderThinkingAndTextBeforeFunctionCall(t *testing.T) {
cache.ClearSignatureCache("")
validSignature := "abc123validSignature1234567890123456789012345678901234567890"
thinkingText := "Let me think about this..."
inputJSON := []byte(`{
"model": "claude-sonnet-4-5-thinking",
"messages": [
{
"role": "user",
"content": [{"type": "text", "text": "Hello"}]
},
{
"role": "assistant",
"content": [
{"type": "text", "text": "Before thinking"},
{"type": "thinking", "thinking": "` + thinkingText + `", "signature": "` + validSignature + `"},
{
"type": "tool_use",
"id": "call_xyz",
"name": "Bash",
"input": {"command": "ls"}
},
{"type": "text", "text": "After tool call"}
]
}
]
}`)
cache.CacheSignature("claude-sonnet-4-5-thinking", thinkingText, validSignature)
output := ConvertClaudeRequestToAntigravity("claude-sonnet-4-5-thinking", inputJSON, false)
outputStr := string(output)
// contents.1 = assistant message (contents.0 = user)
parts := gjson.Get(outputStr, "request.contents.1.parts").Array()
if len(parts) != 4 {
t.Fatalf("Expected 4 parts, got %d", len(parts))
}
// Order: thinking → text → text → functionCall
if !parts[0].Get("thought").Bool() {
t.Error("First part should be thinking")
}
if parts[1].Get("functionCall").Exists() || parts[1].Get("thought").Bool() {
t.Errorf("Second part should be text, got %s", parts[1].Raw)
}
if parts[2].Get("functionCall").Exists() || parts[2].Get("thought").Bool() {
t.Errorf("Third part should be text, got %s", parts[2].Raw)
}
if !parts[3].Get("functionCall").Exists() {
t.Errorf("Last part should be functionCall, got %s", parts[3].Raw)
}
}
func TestConvertClaudeRequestToAntigravity_ToolResult(t *testing.T) {
inputJSON := []byte(`{
"model": "claude-3-5-sonnet-20240620",
@@ -44,6 +44,10 @@ type Params struct {
// Signature caching support
CurrentThinkingText strings.Builder // Accumulates thinking text for signature caching
// Reverse map: sanitized Gemini function name → original Claude tool name.
// Populated lazily on the first response chunk from the original request JSON.
ToolNameMap map[string]string
}
// toolUseIDCounter provides a process-wide unique counter for tool use identifiers.
@@ -71,6 +75,7 @@ func ConvertAntigravityResponseToClaude(_ context.Context, _ string, originalReq
HasFirstResponse: false,
ResponseType: 0,
ResponseIndex: 0,
ToolNameMap: util.SanitizedToolNameMap(originalRequestRawJSON),
}
}
modelName := gjson.GetBytes(requestRawJSON, "model").String()
@@ -212,7 +217,7 @@ func ConvertAntigravityResponseToClaude(_ context.Context, _ string, originalReq
// Handle function/tool calls from the AI model
// This processes tool usage requests and formats them for Claude Code API compatibility
params.HasToolUse = true
fcName := functionCallResult.Get("name").String()
fcName := util.RestoreSanitizedToolName(params.ToolNameMap, functionCallResult.Get("name").String())
// Handle state transitions when switching to function calls
// Close any existing function call block first
@@ -348,7 +353,7 @@ func resolveStopReason(params *Params) string {
// Returns:
// - []byte: A Claude-compatible JSON response.
func ConvertAntigravityResponseToClaudeNonStream(_ context.Context, _ string, originalRequestRawJSON, requestRawJSON, rawJSON []byte, _ *any) []byte {
_ = originalRequestRawJSON
toolNameMap := util.SanitizedToolNameMap(originalRequestRawJSON)
modelName := gjson.GetBytes(requestRawJSON, "model").String()
root := gjson.ParseBytes(rawJSON)
@@ -450,7 +455,7 @@ func ConvertAntigravityResponseToClaudeNonStream(_ context.Context, _ string, or
flushText()
hasToolCall = true
name := functionCall.Get("name").String()
name := util.RestoreSanitizedToolName(toolNameMap, functionCall.Get("name").String())
toolIDCounter++
toolBlock := []byte(`{"type":"tool_use","id":"","name":"","input":{}}`)
toolBlock, _ = sjson.SetBytes(toolBlock, "id", fmt.Sprintf("tool_%d", toolIDCounter))
@@ -286,7 +286,7 @@ func ConvertOpenAIRequestToAntigravity(modelName string, inputRawJSON []byte, _
continue
}
fid := tc.Get("id").String()
fname := tc.Get("function.name").String()
fname := util.SanitizeFunctionName(tc.Get("function.name").String())
fargs := tc.Get("function.arguments").String()
node, _ = sjson.SetBytes(node, "parts."+itoa(p)+".functionCall.id", fid)
node, _ = sjson.SetBytes(node, "parts."+itoa(p)+".functionCall.name", fname)
@@ -309,7 +309,7 @@ func ConvertOpenAIRequestToAntigravity(modelName string, inputRawJSON []byte, _
for _, fid := range fIDs {
if name, ok := tcID2Name[fid]; ok {
toolNode, _ = sjson.SetBytes(toolNode, "parts."+itoa(pp)+".functionResponse.id", fid)
toolNode, _ = sjson.SetBytes(toolNode, "parts."+itoa(pp)+".functionResponse.name", name)
toolNode, _ = sjson.SetBytes(toolNode, "parts."+itoa(pp)+".functionResponse.name", util.SanitizeFunctionName(name))
resp := toolResponses[fid]
if resp == "" {
resp = "{}"
@@ -384,7 +384,9 @@ func ConvertOpenAIRequestToAntigravity(modelName string, inputRawJSON []byte, _
}
fnRaw = string(fnRawBytes)
}
fnRaw, _ = sjson.Delete(fnRaw, "strict")
fnRawBytes := []byte(fnRaw)
fnRawBytes, _ = sjson.SetBytes(fnRawBytes, "name", util.SanitizeFunctionName(fn.Get("name").String()))
fnRaw, _ = sjson.Delete(string(fnRawBytes), "strict")
if !hasFunction {
functionToolNode, _ = sjson.SetRawBytes(functionToolNode, "functionDeclarations", []byte("[]"))
}
@@ -13,6 +13,7 @@ import (
"sync/atomic"
"time"
"github.com/router-for-me/CLIProxyAPI/v6/internal/util"
log "github.com/sirupsen/logrus"
. "github.com/router-for-me/CLIProxyAPI/v6/internal/translator/gemini/openai/chat-completions"
@@ -26,6 +27,7 @@ type convertCliResponseToOpenAIChatParams struct {
FunctionIndex int
SawToolCall bool // Tracks if any tool call was seen in the entire stream
UpstreamFinishReason string // Caches the upstream finish reason for final chunk
SanitizedNameMap map[string]string
}
// functionCallIDCounter provides a process-wide unique counter for function call identifiers.
@@ -48,10 +50,14 @@ var functionCallIDCounter uint64
func ConvertAntigravityResponseToOpenAI(_ context.Context, _ string, originalRequestRawJSON, requestRawJSON, rawJSON []byte, param *any) [][]byte {
if *param == nil {
*param = &convertCliResponseToOpenAIChatParams{
UnixTimestamp: 0,
FunctionIndex: 0,
UnixTimestamp: 0,
FunctionIndex: 0,
SanitizedNameMap: util.SanitizedToolNameMap(originalRequestRawJSON),
}
}
if (*param).(*convertCliResponseToOpenAIChatParams).SanitizedNameMap == nil {
(*param).(*convertCliResponseToOpenAIChatParams).SanitizedNameMap = util.SanitizedToolNameMap(originalRequestRawJSON)
}
if bytes.Equal(rawJSON, []byte("[DONE]")) {
return [][]byte{}
@@ -159,7 +165,7 @@ func ConvertAntigravityResponseToOpenAI(_ context.Context, _ string, originalReq
}
functionCallTemplate := []byte(`{"id": "","index": 0,"type": "function","function": {"name": "","arguments": ""}}`)
fcName := functionCallResult.Get("name").String()
fcName := util.RestoreSanitizedToolName((*param).(*convertCliResponseToOpenAIChatParams).SanitizedNameMap, functionCallResult.Get("name").String())
functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "id", fmt.Sprintf("%s-%d-%d", fcName, time.Now().UnixNano(), atomic.AddUint64(&functionCallIDCounter, 1)))
functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "index", functionCallIndex)
functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "function.name", fcName)
@@ -165,29 +165,22 @@ func ConvertOpenAIRequestToClaude(modelName string, inputRawJSON []byte, stream
// Process messages and transform them to Claude Code format
if messages := root.Get("messages"); messages.Exists() && messages.IsArray() {
messageIndex := 0
systemMessageIndex := -1
messages.ForEach(func(_, message gjson.Result) bool {
role := message.Get("role").String()
contentResult := message.Get("content")
switch role {
case "system":
if systemMessageIndex == -1 {
systemMsg := []byte(`{"role":"user","content":[]}`)
out, _ = sjson.SetRawBytes(out, "messages.-1", systemMsg)
systemMessageIndex = messageIndex
messageIndex++
}
if contentResult.Exists() && contentResult.Type == gjson.String && contentResult.String() != "" {
textPart := []byte(`{"type":"text","text":""}`)
textPart, _ = sjson.SetBytes(textPart, "text", contentResult.String())
out, _ = sjson.SetRawBytes(out, fmt.Sprintf("messages.%d.content.-1", systemMessageIndex), textPart)
out, _ = sjson.SetRawBytes(out, "system.-1", textPart)
} else if contentResult.Exists() && contentResult.IsArray() {
contentResult.ForEach(func(_, part gjson.Result) bool {
if part.Get("type").String() == "text" {
textPart := []byte(`{"type":"text","text":""}`)
textPart, _ = sjson.SetBytes(textPart, "text", part.Get("text").String())
out, _ = sjson.SetRawBytes(out, fmt.Sprintf("messages.%d.content.-1", systemMessageIndex), textPart)
out, _ = sjson.SetRawBytes(out, "system.-1", textPart)
}
return true
})
@@ -269,6 +262,16 @@ func ConvertOpenAIRequestToClaude(modelName string, inputRawJSON []byte, stream
}
return true
})
// Preserve a minimal conversational turn for system-only inputs.
// Claude payloads with top-level system instructions but no messages are risky for downstream validation.
if messageIndex == 0 {
system := gjson.GetBytes(out, "system")
if system.Exists() && system.IsArray() && len(system.Array()) > 0 {
fallbackMsg := []byte(`{"role":"user","content":[{"type":"text","text":""}]}`)
out, _ = sjson.SetRawBytes(out, "messages.-1", fallbackMsg)
}
}
}
// Tools mapping: OpenAI tools -> Claude Code tools
@@ -135,3 +135,111 @@ func TestConvertOpenAIRequestToClaude_ToolResultURLImageOnly(t *testing.T) {
t.Fatalf("Unexpected image URL: %q", got)
}
}
func TestConvertOpenAIRequestToClaude_SystemRoleBecomesTopLevelSystem(t *testing.T) {
inputJSON := `{
"model": "gpt-4.1",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello"}
]
}`
result := ConvertOpenAIRequestToClaude("claude-sonnet-4-5", []byte(inputJSON), false)
resultJSON := gjson.ParseBytes(result)
system := resultJSON.Get("system")
if !system.IsArray() {
t.Fatalf("Expected top-level system array, got %s", system.Raw)
}
if len(system.Array()) != 1 {
t.Fatalf("Expected 1 system block, got %d. System: %s", len(system.Array()), system.Raw)
}
if got := system.Get("0.type").String(); got != "text" {
t.Fatalf("Expected system block type %q, got %q", "text", got)
}
if got := system.Get("0.text").String(); got != "You are a helpful assistant." {
t.Fatalf("Expected system text %q, got %q", "You are a helpful assistant.", got)
}
messages := resultJSON.Get("messages").Array()
if len(messages) != 1 {
t.Fatalf("Expected 1 non-system message, got %d. Messages: %s", len(messages), resultJSON.Get("messages").Raw)
}
if got := messages[0].Get("role").String(); got != "user" {
t.Fatalf("Expected remaining message role %q, got %q", "user", got)
}
if got := messages[0].Get("content.0.text").String(); got != "Hello" {
t.Fatalf("Expected user text %q, got %q", "Hello", got)
}
}
func TestConvertOpenAIRequestToClaude_MultipleSystemMessagesMergedIntoTopLevelSystem(t *testing.T) {
inputJSON := `{
"model": "gpt-4.1",
"messages": [
{"role": "system", "content": "Rule 1"},
{"role": "system", "content": [{"type": "text", "text": "Rule 2"}]},
{"role": "user", "content": "Hello"}
]
}`
result := ConvertOpenAIRequestToClaude("claude-sonnet-4-5", []byte(inputJSON), false)
resultJSON := gjson.ParseBytes(result)
system := resultJSON.Get("system").Array()
if len(system) != 2 {
t.Fatalf("Expected 2 system blocks, got %d. System: %s", len(system), resultJSON.Get("system").Raw)
}
if got := system[0].Get("text").String(); got != "Rule 1" {
t.Fatalf("Expected first system text %q, got %q", "Rule 1", got)
}
if got := system[1].Get("text").String(); got != "Rule 2" {
t.Fatalf("Expected second system text %q, got %q", "Rule 2", got)
}
messages := resultJSON.Get("messages").Array()
if len(messages) != 1 {
t.Fatalf("Expected 1 non-system message, got %d. Messages: %s", len(messages), resultJSON.Get("messages").Raw)
}
if got := messages[0].Get("role").String(); got != "user" {
t.Fatalf("Expected remaining message role %q, got %q", "user", got)
}
if got := messages[0].Get("content.0.text").String(); got != "Hello" {
t.Fatalf("Expected user text %q, got %q", "Hello", got)
}
}
func TestConvertOpenAIRequestToClaude_SystemOnlyInputKeepsFallbackUserMessage(t *testing.T) {
inputJSON := `{
"model": "gpt-4.1",
"messages": [
{"role": "system", "content": "You are a helpful assistant."}
]
}`
result := ConvertOpenAIRequestToClaude("claude-sonnet-4-5", []byte(inputJSON), false)
resultJSON := gjson.ParseBytes(result)
system := resultJSON.Get("system").Array()
if len(system) != 1 {
t.Fatalf("Expected 1 system block, got %d. System: %s", len(system), resultJSON.Get("system").Raw)
}
if got := system[0].Get("text").String(); got != "You are a helpful assistant." {
t.Fatalf("Expected system text %q, got %q", "You are a helpful assistant.", got)
}
messages := resultJSON.Get("messages").Array()
if len(messages) != 1 {
t.Fatalf("Expected 1 fallback message, got %d. Messages: %s", len(messages), resultJSON.Get("messages").Raw)
}
if got := messages[0].Get("role").String(); got != "user" {
t.Fatalf("Expected fallback message role %q, got %q", "user", got)
}
if got := messages[0].Get("content.0.type").String(); got != "text" {
t.Fatalf("Expected fallback content type %q, got %q", "text", got)
}
if got := messages[0].Get("content.0.text").String(); got != "" {
t.Fatalf("Expected fallback text %q, got %q", "", got)
}
}
@@ -36,6 +36,18 @@ type ToolCallAccumulator struct {
Arguments strings.Builder
}
func calculateClaudeUsageTokens(usage gjson.Result) (promptTokens, completionTokens, totalTokens, cachedTokens int64) {
inputTokens := usage.Get("input_tokens").Int()
completionTokens = usage.Get("output_tokens").Int()
cachedTokens = usage.Get("cache_read_input_tokens").Int()
cacheCreationInputTokens := usage.Get("cache_creation_input_tokens").Int()
promptTokens = inputTokens + cacheCreationInputTokens + cachedTokens
totalTokens = promptTokens + completionTokens
return promptTokens, completionTokens, totalTokens, cachedTokens
}
// ConvertClaudeResponseToOpenAI converts Claude Code streaming response format to OpenAI Chat Completions format.
// This function processes various Claude Code event types and transforms them into OpenAI-compatible JSON responses.
// It handles text content, tool calls, reasoning content, and usage metadata, outputting responses that match
@@ -203,14 +215,11 @@ func ConvertClaudeResponseToOpenAI(_ context.Context, modelName string, original
// Handle usage information for token counts
if usage := root.Get("usage"); usage.Exists() {
inputTokens := usage.Get("input_tokens").Int()
outputTokens := usage.Get("output_tokens").Int()
cacheReadInputTokens := usage.Get("cache_read_input_tokens").Int()
cacheCreationInputTokens := usage.Get("cache_creation_input_tokens").Int()
template, _ = sjson.SetBytes(template, "usage.prompt_tokens", inputTokens+cacheCreationInputTokens)
template, _ = sjson.SetBytes(template, "usage.completion_tokens", outputTokens)
template, _ = sjson.SetBytes(template, "usage.total_tokens", inputTokens+outputTokens)
template, _ = sjson.SetBytes(template, "usage.prompt_tokens_details.cached_tokens", cacheReadInputTokens)
promptTokens, completionTokens, totalTokens, cachedTokens := calculateClaudeUsageTokens(usage)
template, _ = sjson.SetBytes(template, "usage.prompt_tokens", promptTokens)
template, _ = sjson.SetBytes(template, "usage.completion_tokens", completionTokens)
template, _ = sjson.SetBytes(template, "usage.total_tokens", totalTokens)
template, _ = sjson.SetBytes(template, "usage.prompt_tokens_details.cached_tokens", cachedTokens)
}
return [][]byte{template}
@@ -362,14 +371,11 @@ func ConvertClaudeResponseToOpenAINonStream(_ context.Context, _ string, origina
}
}
if usage := root.Get("usage"); usage.Exists() {
inputTokens := usage.Get("input_tokens").Int()
outputTokens := usage.Get("output_tokens").Int()
cacheReadInputTokens := usage.Get("cache_read_input_tokens").Int()
cacheCreationInputTokens := usage.Get("cache_creation_input_tokens").Int()
out, _ = sjson.SetBytes(out, "usage.prompt_tokens", inputTokens+cacheCreationInputTokens)
out, _ = sjson.SetBytes(out, "usage.completion_tokens", outputTokens)
out, _ = sjson.SetBytes(out, "usage.total_tokens", inputTokens+outputTokens)
out, _ = sjson.SetBytes(out, "usage.prompt_tokens_details.cached_tokens", cacheReadInputTokens)
promptTokens, completionTokens, totalTokens, cachedTokens := calculateClaudeUsageTokens(usage)
out, _ = sjson.SetBytes(out, "usage.prompt_tokens", promptTokens)
out, _ = sjson.SetBytes(out, "usage.completion_tokens", completionTokens)
out, _ = sjson.SetBytes(out, "usage.total_tokens", totalTokens)
out, _ = sjson.SetBytes(out, "usage.prompt_tokens_details.cached_tokens", cachedTokens)
}
}
}
@@ -0,0 +1,58 @@
package chat_completions
import (
"context"
"testing"
"github.com/tidwall/gjson"
)
func TestConvertClaudeResponseToOpenAI_StreamUsageIncludesCachedTokens(t *testing.T) {
ctx := context.Background()
var param any
out := ConvertClaudeResponseToOpenAI(
ctx,
"claude-opus-4-6",
nil,
nil,
[]byte(`data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"input_tokens":13,"output_tokens":4,"cache_read_input_tokens":22000,"cache_creation_input_tokens":31}}`),
&param,
)
if len(out) != 1 {
t.Fatalf("expected 1 chunk, got %d", len(out))
}
if gotPromptTokens := gjson.GetBytes(out[0], "usage.prompt_tokens").Int(); gotPromptTokens != 22044 {
t.Fatalf("expected prompt_tokens %d, got %d", 22044, gotPromptTokens)
}
if gotCompletionTokens := gjson.GetBytes(out[0], "usage.completion_tokens").Int(); gotCompletionTokens != 4 {
t.Fatalf("expected completion_tokens %d, got %d", 4, gotCompletionTokens)
}
if gotTotalTokens := gjson.GetBytes(out[0], "usage.total_tokens").Int(); gotTotalTokens != 22048 {
t.Fatalf("expected total_tokens %d, got %d", 22048, gotTotalTokens)
}
if gotCachedTokens := gjson.GetBytes(out[0], "usage.prompt_tokens_details.cached_tokens").Int(); gotCachedTokens != 22000 {
t.Fatalf("expected cached_tokens %d, got %d", 22000, gotCachedTokens)
}
}
func TestConvertClaudeResponseToOpenAINonStream_UsageIncludesCachedTokens(t *testing.T) {
rawJSON := []byte("data: {\"type\":\"message_start\",\"message\":{\"id\":\"msg_123\",\"model\":\"claude-opus-4-6\"}}\n" +
"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"},\"usage\":{\"input_tokens\":13,\"output_tokens\":4,\"cache_read_input_tokens\":22000,\"cache_creation_input_tokens\":31}}\n")
out := ConvertClaudeResponseToOpenAINonStream(context.Background(), "", nil, nil, rawJSON, nil)
if gotPromptTokens := gjson.GetBytes(out, "usage.prompt_tokens").Int(); gotPromptTokens != 22044 {
t.Fatalf("expected prompt_tokens %d, got %d", 22044, gotPromptTokens)
}
if gotCompletionTokens := gjson.GetBytes(out, "usage.completion_tokens").Int(); gotCompletionTokens != 4 {
t.Fatalf("expected completion_tokens %d, got %d", 4, gotCompletionTokens)
}
if gotTotalTokens := gjson.GetBytes(out, "usage.total_tokens").Int(); gotTotalTokens != 22048 {
t.Fatalf("expected total_tokens %d, got %d", 22048, gotTotalTokens)
}
if gotCachedTokens := gjson.GetBytes(out, "usage.prompt_tokens_details.cached_tokens").Int(); gotCachedTokens != 22000 {
t.Fatalf("expected cached_tokens %d, got %d", 22000, gotCachedTokens)
}
}
@@ -60,7 +60,7 @@ func ConvertCodexResponseToOpenAI(_ context.Context, modelName string, originalR
rawJSON = bytes.TrimSpace(rawJSON[5:])
// Initialize the OpenAI SSE template.
template := []byte(`{"id":"","object":"chat.completion.chunk","created":12345,"model":"model","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"finish_reason":null,"native_finish_reason":null}]}`)
template := []byte(`{"id":"","object":"chat.completion.chunk","created":12345,"model":"model","choices":[{"index":0,"delta":{},"finish_reason":null,"native_finish_reason":null}]}`)
rootResult := gjson.ParseBytes(rawJSON)
@@ -45,3 +45,48 @@ func TestConvertCodexResponseToOpenAI_FirstChunkUsesRequestModelName(t *testing.
t.Fatalf("expected model %q, got %q", modelName, gotModel)
}
}
func TestConvertCodexResponseToOpenAI_ToolCallChunkOmitsNullContentFields(t *testing.T) {
ctx := context.Background()
var param any
out := ConvertCodexResponseToOpenAI(ctx, "gpt-5.4", nil, nil, []byte(`data: {"type":"response.output_item.added","item":{"type":"function_call","call_id":"call_123","name":"websearch"}}`), &param)
if len(out) != 1 {
t.Fatalf("expected 1 chunk, got %d", len(out))
}
if gjson.GetBytes(out[0], "choices.0.delta.content").Exists() {
t.Fatalf("expected content to be omitted, got %s", string(out[0]))
}
if gjson.GetBytes(out[0], "choices.0.delta.reasoning_content").Exists() {
t.Fatalf("expected reasoning_content to be omitted, got %s", string(out[0]))
}
if !gjson.GetBytes(out[0], "choices.0.delta.tool_calls").Exists() {
t.Fatalf("expected tool_calls to exist, got %s", string(out[0]))
}
}
func TestConvertCodexResponseToOpenAI_ToolCallArgumentsDeltaOmitsNullContentFields(t *testing.T) {
ctx := context.Background()
var param any
out := ConvertCodexResponseToOpenAI(ctx, "gpt-5.4", nil, nil, []byte(`data: {"type":"response.output_item.added","item":{"type":"function_call","call_id":"call_123","name":"websearch"}}`), &param)
if len(out) != 1 {
t.Fatalf("expected tool call announcement chunk, got %d", len(out))
}
out = ConvertCodexResponseToOpenAI(ctx, "gpt-5.4", nil, nil, []byte(`data: {"type":"response.function_call_arguments.delta","delta":"{\"query\":\"OpenAI\"}"}`), &param)
if len(out) != 1 {
t.Fatalf("expected 1 chunk, got %d", len(out))
}
if gjson.GetBytes(out[0], "choices.0.delta.content").Exists() {
t.Fatalf("expected content to be omitted, got %s", string(out[0]))
}
if gjson.GetBytes(out[0], "choices.0.delta.reasoning_content").Exists() {
t.Fatalf("expected reasoning_content to be omitted, got %s", string(out[0]))
}
if !gjson.GetBytes(out[0], "choices.0.delta.tool_calls.0.function.arguments").Exists() {
t.Fatalf("expected tool call arguments delta to exist, got %s", string(out[0]))
}
}
@@ -3,6 +3,7 @@ package responses
import (
"fmt"
log "github.com/sirupsen/logrus"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
)
@@ -39,6 +40,7 @@ func ConvertOpenAIResponsesRequestToCodex(modelName string, inputRawJSON []byte,
// Convert role "system" to "developer" in input array to comply with Codex API requirements.
rawJSON = convertSystemRoleToDeveloper(rawJSON)
rawJSON = normalizeCodexBuiltinTools(rawJSON)
return rawJSON
}
@@ -82,3 +84,59 @@ func convertSystemRoleToDeveloper(rawJSON []byte) []byte {
return result
}
// normalizeCodexBuiltinTools rewrites legacy/preview built-in tool variants to the
// stable names expected by the current Codex upstream.
func normalizeCodexBuiltinTools(rawJSON []byte) []byte {
result := rawJSON
tools := gjson.GetBytes(result, "tools")
if tools.IsArray() {
toolArray := tools.Array()
for i := 0; i < len(toolArray); i++ {
typePath := fmt.Sprintf("tools.%d.type", i)
result = normalizeCodexBuiltinToolAtPath(result, typePath)
}
}
result = normalizeCodexBuiltinToolAtPath(result, "tool_choice.type")
toolChoiceTools := gjson.GetBytes(result, "tool_choice.tools")
if toolChoiceTools.IsArray() {
toolArray := toolChoiceTools.Array()
for i := 0; i < len(toolArray); i++ {
typePath := fmt.Sprintf("tool_choice.tools.%d.type", i)
result = normalizeCodexBuiltinToolAtPath(result, typePath)
}
}
return result
}
func normalizeCodexBuiltinToolAtPath(rawJSON []byte, path string) []byte {
currentType := gjson.GetBytes(rawJSON, path).String()
normalizedType := normalizeCodexBuiltinToolType(currentType)
if normalizedType == "" {
return rawJSON
}
updated, err := sjson.SetBytes(rawJSON, path, normalizedType)
if err != nil {
return rawJSON
}
log.Debugf("codex responses: normalized builtin tool type at %s from %q to %q", path, currentType, normalizedType)
return updated
}
// normalizeCodexBuiltinToolType centralizes the current known Codex Responses
// built-in tool alias compatibility. If Codex introduces more legacy aliases,
// extend this helper instead of adding path-specific rewrite logic elsewhere.
func normalizeCodexBuiltinToolType(toolType string) string {
switch toolType {
case "web_search_preview", "web_search_preview_2025_03_11":
return "web_search"
default:
return ""
}
}
@@ -264,6 +264,52 @@ func TestConvertSystemRoleToDeveloper_AssistantRole(t *testing.T) {
}
}
func TestConvertOpenAIResponsesRequestToCodex_NormalizesWebSearchPreview(t *testing.T) {
inputJSON := []byte(`{
"model": "gpt-5.4-mini",
"input": "find latest OpenAI model news",
"tools": [
{"type": "web_search_preview_2025_03_11"}
],
"tool_choice": {
"type": "allowed_tools",
"tools": [
{"type": "web_search_preview"},
{"type": "web_search_preview_2025_03_11"}
]
}
}`)
output := ConvertOpenAIResponsesRequestToCodex("gpt-5.4-mini", inputJSON, false)
if got := gjson.GetBytes(output, "tools.0.type").String(); got != "web_search" {
t.Fatalf("tools.0.type = %q, want %q: %s", got, "web_search", string(output))
}
if got := gjson.GetBytes(output, "tool_choice.type").String(); got != "allowed_tools" {
t.Fatalf("tool_choice.type = %q, want %q: %s", got, "allowed_tools", string(output))
}
if got := gjson.GetBytes(output, "tool_choice.tools.0.type").String(); got != "web_search" {
t.Fatalf("tool_choice.tools.0.type = %q, want %q: %s", got, "web_search", string(output))
}
if got := gjson.GetBytes(output, "tool_choice.tools.1.type").String(); got != "web_search" {
t.Fatalf("tool_choice.tools.1.type = %q, want %q: %s", got, "web_search", string(output))
}
}
func TestConvertOpenAIResponsesRequestToCodex_NormalizesTopLevelToolChoicePreviewAlias(t *testing.T) {
inputJSON := []byte(`{
"model": "gpt-5.4-mini",
"input": "find latest OpenAI model news",
"tool_choice": {"type": "web_search_preview_2025_03_11"}
}`)
output := ConvertOpenAIResponsesRequestToCodex("gpt-5.4-mini", inputJSON, false)
if got := gjson.GetBytes(output, "tool_choice.type").String(); got != "web_search" {
t.Fatalf("tool_choice.type = %q, want %q: %s", got, "web_search", string(output))
}
}
func TestUserFieldDeletion(t *testing.T) {
inputJSON := []byte(`{
"model": "gpt-5.2",
@@ -89,7 +89,7 @@ func ConvertClaudeRequestToCLI(modelName string, inputRawJSON []byte, _ bool) []
contentJSON, _ = sjson.SetRawBytes(contentJSON, "parts.-1", part)
case "tool_use":
functionName := contentResult.Get("name").String()
functionName := util.SanitizeFunctionName(contentResult.Get("name").String())
functionArgs := contentResult.Get("input").String()
argsResult := gjson.Parse(functionArgs)
if argsResult.IsObject() && gjson.Valid(functionArgs) {
@@ -112,7 +112,7 @@ func ConvertClaudeRequestToCLI(modelName string, inputRawJSON []byte, _ bool) []
}
responseData := contentResult.Get("content").Raw
part := []byte(`{"functionResponse":{"name":"","response":{"result":""}}}`)
part, _ = sjson.SetBytes(part, "functionResponse.name", funcName)
part, _ = sjson.SetBytes(part, "functionResponse.name", util.SanitizeFunctionName(funcName))
part, _ = sjson.SetBytes(part, "functionResponse.response.result", responseData)
contentJSON, _ = sjson.SetRawBytes(contentJSON, "parts.-1", part)
@@ -151,6 +151,7 @@ func ConvertClaudeRequestToCLI(modelName string, inputRawJSON []byte, _ bool) []
inputSchema := util.CleanJSONSchemaForGemini(inputSchemaResult.Raw)
tool, _ := sjson.DeleteBytes([]byte(toolResult.Raw), "input_schema")
tool, _ = sjson.SetRawBytes(tool, "parametersJsonSchema", []byte(inputSchema))
tool, _ = sjson.SetBytes(tool, "name", util.SanitizeFunctionName(gjson.GetBytes(tool, "name").String()))
tool, _ = sjson.DeleteBytes(tool, "strict")
tool, _ = sjson.DeleteBytes(tool, "input_examples")
tool, _ = sjson.DeleteBytes(tool, "type")
@@ -194,7 +195,7 @@ func ConvertClaudeRequestToCLI(modelName string, inputRawJSON []byte, _ bool) []
case "tool":
out, _ = sjson.SetBytes(out, "request.toolConfig.functionCallingConfig.mode", "ANY")
if toolChoiceName != "" {
out, _ = sjson.SetBytes(out, "request.toolConfig.functionCallingConfig.allowedFunctionNames", []string{toolChoiceName})
out, _ = sjson.SetBytes(out, "request.toolConfig.functionCallingConfig.allowedFunctionNames", []string{util.SanitizeFunctionName(toolChoiceName)})
}
}
}
@@ -28,6 +28,9 @@ type Params struct {
ResponseType int // Current response type: 0=none, 1=content, 2=thinking, 3=function
ResponseIndex int // Index counter for content blocks in the streaming response
HasContent bool // Tracks whether any content (text, thinking, or tool use) has been output
// Reverse map: sanitized Gemini function name → original Claude tool name.
ToolNameMap map[string]string
}
// toolUseIDCounter provides a process-wide unique counter for tool use identifiers.
@@ -55,6 +58,7 @@ func ConvertGeminiCLIResponseToClaude(_ context.Context, _ string, originalReque
HasFirstResponse: false,
ResponseType: 0,
ResponseIndex: 0,
ToolNameMap: util.SanitizedToolNameMap(originalRequestRawJSON),
}
}
@@ -165,7 +169,7 @@ func ConvertGeminiCLIResponseToClaude(_ context.Context, _ string, originalReque
// Handle function/tool calls from the AI model
// This processes tool usage requests and formats them for Claude Code API compatibility
usedTool = true
fcName := functionCallResult.Get("name").String()
fcName := util.RestoreSanitizedToolName((*param).(*Params).ToolNameMap, functionCallResult.Get("name").String())
// Handle state transitions when switching to function calls
// Close any existing function call block first
@@ -248,7 +252,7 @@ func ConvertGeminiCLIResponseToClaude(_ context.Context, _ string, originalReque
// Returns:
// - []byte: A Claude-compatible JSON response.
func ConvertGeminiCLIResponseToClaudeNonStream(_ context.Context, _ string, originalRequestRawJSON, requestRawJSON, rawJSON []byte, _ *any) []byte {
_ = originalRequestRawJSON
toolNameMap := util.SanitizedToolNameMap(originalRequestRawJSON)
_ = requestRawJSON
root := gjson.ParseBytes(rawJSON)
@@ -306,7 +310,7 @@ func ConvertGeminiCLIResponseToClaudeNonStream(_ context.Context, _ string, orig
flushText()
hasToolCall = true
name := functionCall.Get("name").String()
name := util.RestoreSanitizedToolName(toolNameMap, functionCall.Get("name").String())
toolIDCounter++
toolBlock := []byte(`{"type":"tool_use","id":"","name":"","input":{}}`)
toolBlock, _ = sjson.SetBytes(toolBlock, "id", fmt.Sprintf("tool_%d", toolIDCounter))
@@ -251,7 +251,7 @@ func ConvertOpenAIRequestToGeminiCLI(modelName string, inputRawJSON []byte, _ bo
continue
}
fid := tc.Get("id").String()
fname := tc.Get("function.name").String()
fname := util.SanitizeFunctionName(tc.Get("function.name").String())
fargs := tc.Get("function.arguments").String()
node, _ = sjson.SetBytes(node, "parts."+itoa(p)+".functionCall.name", fname)
node, _ = sjson.SetRawBytes(node, "parts."+itoa(p)+".functionCall.args", []byte(fargs))
@@ -268,7 +268,7 @@ func ConvertOpenAIRequestToGeminiCLI(modelName string, inputRawJSON []byte, _ bo
pp := 0
for _, fid := range fIDs {
if name, ok := tcID2Name[fid]; ok {
toolNode, _ = sjson.SetBytes(toolNode, "parts."+itoa(pp)+".functionResponse.name", name)
toolNode, _ = sjson.SetBytes(toolNode, "parts."+itoa(pp)+".functionResponse.name", util.SanitizeFunctionName(name))
resp := toolResponses[fid]
if resp == "" {
resp = "{}"
@@ -331,6 +331,7 @@ func ConvertOpenAIRequestToGeminiCLI(modelName string, inputRawJSON []byte, _ bo
continue
}
}
fnRaw, _ = sjson.SetBytes(fnRaw, "name", util.SanitizeFunctionName(fn.Get("name").String()))
fnRaw, _ = sjson.DeleteBytes(fnRaw, "strict")
if !hasFunction {
functionToolNode, _ = sjson.SetRawBytes(functionToolNode, "functionDeclarations", []byte("[]"))
@@ -14,6 +14,7 @@ import (
"time"
. "github.com/router-for-me/CLIProxyAPI/v6/internal/translator/gemini/openai/chat-completions"
"github.com/router-for-me/CLIProxyAPI/v6/internal/util"
log "github.com/sirupsen/logrus"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
@@ -21,8 +22,9 @@ import (
// convertCliResponseToOpenAIChatParams holds parameters for response conversion.
type convertCliResponseToOpenAIChatParams struct {
UnixTimestamp int64
FunctionIndex int
UnixTimestamp int64
FunctionIndex int
SanitizedNameMap map[string]string
}
// functionCallIDCounter provides a process-wide unique counter for function call identifiers.
@@ -45,10 +47,14 @@ var functionCallIDCounter uint64
func ConvertCliResponseToOpenAI(_ context.Context, _ string, originalRequestRawJSON, requestRawJSON, rawJSON []byte, param *any) [][]byte {
if *param == nil {
*param = &convertCliResponseToOpenAIChatParams{
UnixTimestamp: 0,
FunctionIndex: 0,
UnixTimestamp: 0,
FunctionIndex: 0,
SanitizedNameMap: util.SanitizedToolNameMap(originalRequestRawJSON),
}
}
if (*param).(*convertCliResponseToOpenAIChatParams).SanitizedNameMap == nil {
(*param).(*convertCliResponseToOpenAIChatParams).SanitizedNameMap = util.SanitizedToolNameMap(originalRequestRawJSON)
}
if bytes.Equal(rawJSON, []byte("[DONE]")) {
return [][]byte{}
@@ -163,7 +169,7 @@ func ConvertCliResponseToOpenAI(_ context.Context, _ string, originalRequestRawJ
}
functionCallTemplate := []byte(`{"id":"","index":0,"type":"function","function":{"name":"","arguments":""}}`)
fcName := functionCallResult.Get("name").String()
fcName := util.RestoreSanitizedToolName((*param).(*convertCliResponseToOpenAIChatParams).SanitizedNameMap, functionCallResult.Get("name").String())
functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "id", fmt.Sprintf("%s-%d-%d", fcName, time.Now().UnixNano(), atomic.AddUint64(&functionCallIDCounter, 1)))
functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "index", functionCallIndex)
functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "function.name", fcName)
@@ -11,6 +11,7 @@ import (
"github.com/router-for-me/CLIProxyAPI/v6/internal/registry"
"github.com/router-for-me/CLIProxyAPI/v6/internal/translator/gemini/common"
"github.com/router-for-me/CLIProxyAPI/v6/internal/util"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
)
@@ -90,6 +91,7 @@ func ConvertClaudeRequestToGemini(modelName string, inputRawJSON []byte, _ bool)
functionName = derived
}
}
functionName = util.SanitizeFunctionName(functionName)
functionArgs := contentResult.Get("input").String()
argsResult := gjson.Parse(functionArgs)
if argsResult.IsObject() && gjson.Valid(functionArgs) {
@@ -109,6 +111,7 @@ func ConvertClaudeRequestToGemini(modelName string, inputRawJSON []byte, _ bool)
if funcName == "" {
funcName = toolCallID
}
funcName = util.SanitizeFunctionName(funcName)
responseData := contentResult.Get("content").Raw
part := []byte(`{"functionResponse":{"name":"","response":{"result":""}}}`)
part, _ = sjson.SetBytes(part, "functionResponse.name", funcName)
@@ -165,6 +168,7 @@ func ConvertClaudeRequestToGemini(modelName string, inputRawJSON []byte, _ bool)
tool, _ = sjson.DeleteBytes(tool, "type")
tool, _ = sjson.DeleteBytes(tool, "cache_control")
tool, _ = sjson.DeleteBytes(tool, "defer_loading")
tool, _ = sjson.SetBytes(tool, "name", util.SanitizeFunctionName(gjson.GetBytes(tool, "name").String()))
if gjson.ValidBytes(tool) && gjson.ParseBytes(tool).IsObject() {
if !hasTools {
out, _ = sjson.SetRawBytes(out, "tools", []byte(`[{"functionDeclarations":[]}]`))
@@ -202,7 +206,7 @@ func ConvertClaudeRequestToGemini(modelName string, inputRawJSON []byte, _ bool)
case "tool":
out, _ = sjson.SetBytes(out, "toolConfig.functionCallingConfig.mode", "ANY")
if toolChoiceName != "" {
out, _ = sjson.SetBytes(out, "toolConfig.functionCallingConfig.allowedFunctionNames", []string{toolChoiceName})
out, _ = sjson.SetBytes(out, "toolConfig.functionCallingConfig.allowedFunctionNames", []string{util.SanitizeFunctionName(toolChoiceName)})
}
}
}
@@ -27,6 +27,7 @@ type Params struct {
ResponseIndex int
HasContent bool // Tracks whether any content (text, thinking, or tool use) has been output
ToolNameMap map[string]string
SanitizedNameMap map[string]string
SawToolCall bool
}
@@ -57,6 +58,7 @@ func ConvertGeminiResponseToClaude(_ context.Context, _ string, originalRequestR
ResponseType: 0,
ResponseIndex: 0,
ToolNameMap: util.ToolNameMapFromClaudeRequest(originalRequestRawJSON),
SanitizedNameMap: util.SanitizedToolNameMap(originalRequestRawJSON),
SawToolCall: false,
}
}
@@ -167,6 +169,7 @@ func ConvertGeminiResponseToClaude(_ context.Context, _ string, originalRequestR
// This processes tool usage requests and formats them for Claude API compatibility
(*param).(*Params).SawToolCall = true
upstreamToolName := functionCallResult.Get("name").String()
upstreamToolName = util.RestoreSanitizedToolName((*param).(*Params).SanitizedNameMap, upstreamToolName)
clientToolName := util.MapToolName((*param).(*Params).ToolNameMap, upstreamToolName)
// FIX: Handle streaming split/delta where name might be empty in subsequent chunks.
@@ -260,6 +263,7 @@ func ConvertGeminiResponseToClaudeNonStream(_ context.Context, _ string, origina
root := gjson.ParseBytes(rawJSON)
toolNameMap := util.ToolNameMapFromClaudeRequest(originalRequestRawJSON)
sanitizedNameMap := util.SanitizedToolNameMap(originalRequestRawJSON)
out := []byte(`{"id":"","type":"message","role":"assistant","model":"","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":0,"output_tokens":0}}`)
out, _ = sjson.SetBytes(out, "id", root.Get("responseId").String())
@@ -315,6 +319,7 @@ func ConvertGeminiResponseToClaudeNonStream(_ context.Context, _ string, origina
hasToolCall = true
upstreamToolName := functionCall.Get("name").String()
upstreamToolName = util.RestoreSanitizedToolName(sanitizedNameMap, upstreamToolName)
clientToolName := util.MapToolName(toolNameMap, upstreamToolName)
toolIDCounter++
toolBlock := []byte(`{"type":"tool_use","id":"","name":"","input":{}}`)
@@ -257,7 +257,7 @@ func ConvertOpenAIRequestToGemini(modelName string, inputRawJSON []byte, _ bool)
continue
}
fid := tc.Get("id").String()
fname := tc.Get("function.name").String()
fname := util.SanitizeFunctionName(tc.Get("function.name").String())
fargs := tc.Get("function.arguments").String()
node, _ = sjson.SetBytes(node, "parts."+itoa(p)+".functionCall.name", fname)
node, _ = sjson.SetRawBytes(node, "parts."+itoa(p)+".functionCall.args", []byte(fargs))
@@ -274,7 +274,7 @@ func ConvertOpenAIRequestToGemini(modelName string, inputRawJSON []byte, _ bool)
pp := 0
for _, fid := range fIDs {
if name, ok := tcID2Name[fid]; ok {
toolNode, _ = sjson.SetBytes(toolNode, "parts."+itoa(pp)+".functionResponse.name", name)
toolNode, _ = sjson.SetBytes(toolNode, "parts."+itoa(pp)+".functionResponse.name", util.SanitizeFunctionName(name))
resp := toolResponses[fid]
if resp == "" {
resp = "{}"
@@ -341,6 +341,9 @@ func ConvertOpenAIRequestToGemini(modelName string, inputRawJSON []byte, _ bool)
}
fnRaw = string(fnRawBytes)
}
fnRawBytes := []byte(fnRaw)
fnRawBytes, _ = sjson.SetBytes(fnRawBytes, "name", util.SanitizeFunctionName(fn.Get("name").String()))
fnRaw = string(fnRawBytes)
fnRaw, _ = sjson.Delete(fnRaw, "strict")
if !hasFunction {
functionToolNode, _ = sjson.SetRawBytes(functionToolNode, "functionDeclarations", []byte("[]"))
@@ -13,6 +13,7 @@ import (
"sync/atomic"
"time"
"github.com/router-for-me/CLIProxyAPI/v6/internal/util"
log "github.com/sirupsen/logrus"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
@@ -22,7 +23,8 @@ import (
type convertGeminiResponseToOpenAIChatParams struct {
UnixTimestamp int64
// FunctionIndex tracks tool call indices per candidate index to support multiple candidates.
FunctionIndex map[int]int
FunctionIndex map[int]int
SanitizedNameMap map[string]string
}
// functionCallIDCounter provides a process-wide unique counter for function call identifiers.
@@ -46,8 +48,9 @@ func ConvertGeminiResponseToOpenAI(_ context.Context, _ string, originalRequestR
// Initialize parameters if nil.
if *param == nil {
*param = &convertGeminiResponseToOpenAIChatParams{
UnixTimestamp: 0,
FunctionIndex: make(map[int]int),
UnixTimestamp: 0,
FunctionIndex: make(map[int]int),
SanitizedNameMap: util.SanitizedToolNameMap(originalRequestRawJSON),
}
}
@@ -56,6 +59,9 @@ func ConvertGeminiResponseToOpenAI(_ context.Context, _ string, originalRequestR
if p.FunctionIndex == nil {
p.FunctionIndex = make(map[int]int)
}
if p.SanitizedNameMap == nil {
p.SanitizedNameMap = util.SanitizedToolNameMap(originalRequestRawJSON)
}
if bytes.HasPrefix(rawJSON, []byte("data:")) {
rawJSON = bytes.TrimSpace(rawJSON[5:])
@@ -191,7 +197,7 @@ func ConvertGeminiResponseToOpenAI(_ context.Context, _ string, originalRequestR
}
functionCallTemplate := []byte(`{"id":"","index":0,"type":"function","function":{"name":"","arguments":""}}`)
fcName := functionCallResult.Get("name").String()
fcName := util.RestoreSanitizedToolName(p.SanitizedNameMap, functionCallResult.Get("name").String())
functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "id", fmt.Sprintf("%s-%d-%d", fcName, time.Now().UnixNano(), atomic.AddUint64(&functionCallIDCounter, 1)))
functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "index", functionCallIndex)
functionCallTemplate, _ = sjson.SetBytes(functionCallTemplate, "function.name", fcName)
@@ -265,6 +271,7 @@ func ConvertGeminiResponseToOpenAI(_ context.Context, _ string, originalRequestR
// Returns:
// - []byte: An OpenAI-compatible JSON response containing all message content and metadata
func ConvertGeminiResponseToOpenAINonStream(_ context.Context, _ string, originalRequestRawJSON, requestRawJSON, rawJSON []byte, _ *any) []byte {
sanitizedNameMap := util.SanitizedToolNameMap(originalRequestRawJSON)
var unixTimestamp int64
// Initialize template with an empty choices array to support multiple candidates.
template := []byte(`{"id":"","object":"chat.completion","created":123456,"model":"model","choices":[]}`)
@@ -358,7 +365,7 @@ func ConvertGeminiResponseToOpenAINonStream(_ context.Context, _ string, origina
choiceTemplate, _ = sjson.SetRawBytes(choiceTemplate, "message.tool_calls", []byte(`[]`))
}
functionCallItemTemplate := []byte(`{"id":"","type":"function","function":{"name":"","arguments":""}}`)
fcName := functionCallResult.Get("name").String()
fcName := util.RestoreSanitizedToolName(sanitizedNameMap, functionCallResult.Get("name").String())
functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "id", fmt.Sprintf("%s-%d-%d", fcName, time.Now().UnixNano(), atomic.AddUint64(&functionCallIDCounter, 1)))
functionCallItemTemplate, _ = sjson.SetBytes(functionCallItemTemplate, "function.name", fcName)
if fcArgsResult := functionCallResult.Get("args"); fcArgsResult.Exists() {
@@ -5,6 +5,7 @@ import (
"strings"
"github.com/router-for-me/CLIProxyAPI/v6/internal/translator/gemini/common"
"github.com/router-for-me/CLIProxyAPI/v6/internal/util"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
)
@@ -291,7 +292,7 @@ func ConvertOpenAIResponsesRequestToGemini(modelName string, inputRawJSON []byte
case "function_call":
// Handle function calls - convert to model message with functionCall
name := item.Get("name").String()
name := util.SanitizeFunctionName(item.Get("name").String())
arguments := item.Get("arguments").String()
modelContent := []byte(`{"role":"model","parts":[]}`)
@@ -333,6 +334,7 @@ func ConvertOpenAIResponsesRequestToGemini(modelName string, inputRawJSON []byte
return true
})
}
functionName = util.SanitizeFunctionName(functionName)
functionResponse, _ = sjson.SetBytes(functionResponse, "functionResponse.name", functionName)
functionResponse, _ = sjson.SetBytes(functionResponse, "functionResponse.id", callID)
@@ -375,7 +377,7 @@ func ConvertOpenAIResponsesRequestToGemini(modelName string, inputRawJSON []byte
funcDecl := []byte(`{"name":"","description":"","parametersJsonSchema":{}}`)
if name := tool.Get("name"); name.Exists() {
funcDecl, _ = sjson.SetBytes(funcDecl, "name", name.String())
funcDecl, _ = sjson.SetBytes(funcDecl, "name", util.SanitizeFunctionName(name.String()))
}
if desc := tool.Get("description"); desc.Exists() {
funcDecl, _ = sjson.SetBytes(funcDecl, "description", desc.String())
@@ -9,6 +9,7 @@ import (
"time"
translatorcommon "github.com/router-for-me/CLIProxyAPI/v6/internal/translator/common"
"github.com/router-for-me/CLIProxyAPI/v6/internal/util"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
)
@@ -36,11 +37,12 @@ type geminiToResponsesState struct {
ReasoningClosed bool
// function call aggregation (keyed by output_index)
NextIndex int
FuncArgsBuf map[int]*strings.Builder
FuncNames map[int]string
FuncCallIDs map[int]string
FuncDone map[int]bool
NextIndex int
FuncArgsBuf map[int]*strings.Builder
FuncNames map[int]string
FuncCallIDs map[int]string
FuncDone map[int]bool
SanitizedNameMap map[string]string
}
// responseIDCounter provides a process-wide unique counter for synthesized response identifiers.
@@ -90,10 +92,11 @@ func emitEvent(event string, payload []byte) []byte {
func ConvertGeminiResponseToOpenAIResponses(_ context.Context, modelName string, originalRequestRawJSON, requestRawJSON, rawJSON []byte, param *any) [][]byte {
if *param == nil {
*param = &geminiToResponsesState{
FuncArgsBuf: make(map[int]*strings.Builder),
FuncNames: make(map[int]string),
FuncCallIDs: make(map[int]string),
FuncDone: make(map[int]bool),
FuncArgsBuf: make(map[int]*strings.Builder),
FuncNames: make(map[int]string),
FuncCallIDs: make(map[int]string),
FuncDone: make(map[int]bool),
SanitizedNameMap: util.SanitizedToolNameMap(originalRequestRawJSON),
}
}
st := (*param).(*geminiToResponsesState)
@@ -109,6 +112,9 @@ func ConvertGeminiResponseToOpenAIResponses(_ context.Context, modelName string,
if st.FuncDone == nil {
st.FuncDone = make(map[int]bool)
}
if st.SanitizedNameMap == nil {
st.SanitizedNameMap = util.SanitizedToolNameMap(originalRequestRawJSON)
}
if bytes.HasPrefix(rawJSON, []byte("data:")) {
rawJSON = bytes.TrimSpace(rawJSON[5:])
@@ -306,7 +312,7 @@ func ConvertGeminiResponseToOpenAIResponses(_ context.Context, modelName string,
// Responses streaming requires message done events before the next output_item.added.
finalizeReasoning()
finalizeMessage()
name := fc.Get("name").String()
name := util.RestoreSanitizedToolName(st.SanitizedNameMap, fc.Get("name").String())
idx := st.NextIndex
st.NextIndex++
// Ensure buffers
@@ -565,6 +571,7 @@ func ConvertGeminiResponseToOpenAIResponses(_ context.Context, modelName string,
func ConvertGeminiResponseToOpenAIResponsesNonStream(_ context.Context, _ string, originalRequestRawJSON, requestRawJSON, rawJSON []byte, _ *any) []byte {
root := gjson.ParseBytes(rawJSON)
root = unwrapGeminiResponseRoot(root)
sanitizedNameMap := util.SanitizedToolNameMap(originalRequestRawJSON)
// Base response scaffold
resp := []byte(`{"id":"","object":"response","created_at":0,"status":"completed","background":false,"error":null,"incomplete_details":null}`)
@@ -694,7 +701,7 @@ func ConvertGeminiResponseToOpenAIResponsesNonStream(_ context.Context, _ string
return true
}
if fc := p.Get("functionCall"); fc.Exists() {
name := fc.Get("name").String()
name := util.RestoreSanitizedToolName(sanitizedNameMap, fc.Get("name").String())
args := fc.Get("args")
callID := fmt.Sprintf("call_%x_%d", time.Now().UnixNano(), atomic.AddUint64(&funcCallIDCounter, 1))
itemJSON := []byte(`{"id":"","type":"function_call","status":"completed","arguments":"","call_id":"","name":""}`)
@@ -4,6 +4,7 @@ import (
"bytes"
"context"
"fmt"
"sort"
"strings"
"sync/atomic"
"time"
@@ -16,6 +17,7 @@ import (
type oaiToResponsesStateReasoning struct {
ReasoningID string
ReasoningData string
OutputIndex int
}
type oaiToResponsesState struct {
Seq int
@@ -29,16 +31,19 @@ type oaiToResponsesState struct {
MsgTextBuf map[int]*strings.Builder
ReasoningBuf strings.Builder
Reasonings []oaiToResponsesStateReasoning
FuncArgsBuf map[int]*strings.Builder // index -> args
FuncNames map[int]string // index -> name
FuncCallIDs map[int]string // index -> call_id
FuncArgsBuf map[string]*strings.Builder
FuncNames map[string]string
FuncCallIDs map[string]string
FuncOutputIx map[string]int
MsgOutputIx map[int]int
NextOutputIx int
// message item state per output index
MsgItemAdded map[int]bool // whether response.output_item.added emitted for message
MsgContentAdded map[int]bool // whether response.content_part.added emitted for message
MsgItemDone map[int]bool // whether message done events were emitted
// function item done state
FuncArgsDone map[int]bool
FuncItemDone map[int]bool
FuncArgsDone map[string]bool
FuncItemDone map[string]bool
// usage aggregation
PromptTokens int64
CachedTokens int64
@@ -60,15 +65,17 @@ func emitRespEvent(event string, payload []byte) []byte {
func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context, modelName string, originalRequestRawJSON, requestRawJSON, rawJSON []byte, param *any) [][]byte {
if *param == nil {
*param = &oaiToResponsesState{
FuncArgsBuf: make(map[int]*strings.Builder),
FuncNames: make(map[int]string),
FuncCallIDs: make(map[int]string),
FuncArgsBuf: make(map[string]*strings.Builder),
FuncNames: make(map[string]string),
FuncCallIDs: make(map[string]string),
FuncOutputIx: make(map[string]int),
MsgOutputIx: make(map[int]int),
MsgTextBuf: make(map[int]*strings.Builder),
MsgItemAdded: make(map[int]bool),
MsgContentAdded: make(map[int]bool),
MsgItemDone: make(map[int]bool),
FuncArgsDone: make(map[int]bool),
FuncItemDone: make(map[int]bool),
FuncArgsDone: make(map[string]bool),
FuncItemDone: make(map[string]bool),
Reasonings: make([]oaiToResponsesStateReasoning, 0),
}
}
@@ -125,6 +132,12 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
}
nextSeq := func() int { st.Seq++; return st.Seq }
allocOutputIndex := func() int {
ix := st.NextOutputIx
st.NextOutputIx++
return ix
}
toolStateKey := func(outputIndex, toolIndex int) string { return fmt.Sprintf("%d:%d", outputIndex, toolIndex) }
var out [][]byte
if !st.Started {
@@ -135,14 +148,17 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
st.ReasoningBuf.Reset()
st.ReasoningID = ""
st.ReasoningIndex = 0
st.FuncArgsBuf = make(map[int]*strings.Builder)
st.FuncNames = make(map[int]string)
st.FuncCallIDs = make(map[int]string)
st.FuncArgsBuf = make(map[string]*strings.Builder)
st.FuncNames = make(map[string]string)
st.FuncCallIDs = make(map[string]string)
st.FuncOutputIx = make(map[string]int)
st.MsgOutputIx = make(map[int]int)
st.NextOutputIx = 0
st.MsgItemAdded = make(map[int]bool)
st.MsgContentAdded = make(map[int]bool)
st.MsgItemDone = make(map[int]bool)
st.FuncArgsDone = make(map[int]bool)
st.FuncItemDone = make(map[int]bool)
st.FuncArgsDone = make(map[string]bool)
st.FuncItemDone = make(map[string]bool)
st.PromptTokens = 0
st.CachedTokens = 0
st.CompletionTokens = 0
@@ -185,7 +201,7 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
outputItemDone, _ = sjson.SetBytes(outputItemDone, "item.summary.text", text)
out = append(out, emitRespEvent("response.output_item.done", outputItemDone))
st.Reasonings = append(st.Reasonings, oaiToResponsesStateReasoning{ReasoningID: st.ReasoningID, ReasoningData: text})
st.Reasonings = append(st.Reasonings, oaiToResponsesStateReasoning{ReasoningID: st.ReasoningID, ReasoningData: text, OutputIndex: st.ReasoningIndex})
st.ReasoningID = ""
}
@@ -201,10 +217,14 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
stopReasoning(st.ReasoningBuf.String())
st.ReasoningBuf.Reset()
}
if _, exists := st.MsgOutputIx[idx]; !exists {
st.MsgOutputIx[idx] = allocOutputIndex()
}
msgOutputIndex := st.MsgOutputIx[idx]
if !st.MsgItemAdded[idx] {
item := []byte(`{"type":"response.output_item.added","sequence_number":0,"output_index":0,"item":{"id":"","type":"message","status":"in_progress","content":[],"role":"assistant"}}`)
item, _ = sjson.SetBytes(item, "sequence_number", nextSeq())
item, _ = sjson.SetBytes(item, "output_index", idx)
item, _ = sjson.SetBytes(item, "output_index", msgOutputIndex)
item, _ = sjson.SetBytes(item, "item.id", fmt.Sprintf("msg_%s_%d", st.ResponseID, idx))
out = append(out, emitRespEvent("response.output_item.added", item))
st.MsgItemAdded[idx] = true
@@ -213,7 +233,7 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
part := []byte(`{"type":"response.content_part.added","sequence_number":0,"item_id":"","output_index":0,"content_index":0,"part":{"type":"output_text","annotations":[],"logprobs":[],"text":""}}`)
part, _ = sjson.SetBytes(part, "sequence_number", nextSeq())
part, _ = sjson.SetBytes(part, "item_id", fmt.Sprintf("msg_%s_%d", st.ResponseID, idx))
part, _ = sjson.SetBytes(part, "output_index", idx)
part, _ = sjson.SetBytes(part, "output_index", msgOutputIndex)
part, _ = sjson.SetBytes(part, "content_index", 0)
out = append(out, emitRespEvent("response.content_part.added", part))
st.MsgContentAdded[idx] = true
@@ -222,7 +242,7 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
msg := []byte(`{"type":"response.output_text.delta","sequence_number":0,"item_id":"","output_index":0,"content_index":0,"delta":"","logprobs":[]}`)
msg, _ = sjson.SetBytes(msg, "sequence_number", nextSeq())
msg, _ = sjson.SetBytes(msg, "item_id", fmt.Sprintf("msg_%s_%d", st.ResponseID, idx))
msg, _ = sjson.SetBytes(msg, "output_index", idx)
msg, _ = sjson.SetBytes(msg, "output_index", msgOutputIndex)
msg, _ = sjson.SetBytes(msg, "content_index", 0)
msg, _ = sjson.SetBytes(msg, "delta", c.String())
out = append(out, emitRespEvent("response.output_text.delta", msg))
@@ -238,10 +258,10 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
// On first appearance, add reasoning item and part
if st.ReasoningID == "" {
st.ReasoningID = fmt.Sprintf("rs_%s_%d", st.ResponseID, idx)
st.ReasoningIndex = idx
st.ReasoningIndex = allocOutputIndex()
item := []byte(`{"type":"response.output_item.added","sequence_number":0,"output_index":0,"item":{"id":"","type":"reasoning","status":"in_progress","summary":[]}}`)
item, _ = sjson.SetBytes(item, "sequence_number", nextSeq())
item, _ = sjson.SetBytes(item, "output_index", idx)
item, _ = sjson.SetBytes(item, "output_index", st.ReasoningIndex)
item, _ = sjson.SetBytes(item, "item.id", st.ReasoningID)
out = append(out, emitRespEvent("response.output_item.added", item))
part := []byte(`{"type":"response.reasoning_summary_part.added","sequence_number":0,"item_id":"","output_index":0,"summary_index":0,"part":{"type":"summary_text","text":""}}`)
@@ -269,6 +289,7 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
// Before emitting any function events, if a message is open for this index,
// close its text/content to match Codex expected ordering.
if st.MsgItemAdded[idx] && !st.MsgItemDone[idx] {
msgOutputIndex := st.MsgOutputIx[idx]
fullText := ""
if b := st.MsgTextBuf[idx]; b != nil {
fullText = b.String()
@@ -276,7 +297,7 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
done := []byte(`{"type":"response.output_text.done","sequence_number":0,"item_id":"","output_index":0,"content_index":0,"text":"","logprobs":[]}`)
done, _ = sjson.SetBytes(done, "sequence_number", nextSeq())
done, _ = sjson.SetBytes(done, "item_id", fmt.Sprintf("msg_%s_%d", st.ResponseID, idx))
done, _ = sjson.SetBytes(done, "output_index", idx)
done, _ = sjson.SetBytes(done, "output_index", msgOutputIndex)
done, _ = sjson.SetBytes(done, "content_index", 0)
done, _ = sjson.SetBytes(done, "text", fullText)
out = append(out, emitRespEvent("response.output_text.done", done))
@@ -284,69 +305,72 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
partDone := []byte(`{"type":"response.content_part.done","sequence_number":0,"item_id":"","output_index":0,"content_index":0,"part":{"type":"output_text","annotations":[],"logprobs":[],"text":""}}`)
partDone, _ = sjson.SetBytes(partDone, "sequence_number", nextSeq())
partDone, _ = sjson.SetBytes(partDone, "item_id", fmt.Sprintf("msg_%s_%d", st.ResponseID, idx))
partDone, _ = sjson.SetBytes(partDone, "output_index", idx)
partDone, _ = sjson.SetBytes(partDone, "output_index", msgOutputIndex)
partDone, _ = sjson.SetBytes(partDone, "content_index", 0)
partDone, _ = sjson.SetBytes(partDone, "part.text", fullText)
out = append(out, emitRespEvent("response.content_part.done", partDone))
itemDone := []byte(`{"type":"response.output_item.done","sequence_number":0,"output_index":0,"item":{"id":"","type":"message","status":"completed","content":[{"type":"output_text","annotations":[],"logprobs":[],"text":""}],"role":"assistant"}}`)
itemDone, _ = sjson.SetBytes(itemDone, "sequence_number", nextSeq())
itemDone, _ = sjson.SetBytes(itemDone, "output_index", idx)
itemDone, _ = sjson.SetBytes(itemDone, "output_index", msgOutputIndex)
itemDone, _ = sjson.SetBytes(itemDone, "item.id", fmt.Sprintf("msg_%s_%d", st.ResponseID, idx))
itemDone, _ = sjson.SetBytes(itemDone, "item.content.0.text", fullText)
out = append(out, emitRespEvent("response.output_item.done", itemDone))
st.MsgItemDone[idx] = true
}
// Only emit item.added once per tool call and preserve call_id across chunks.
newCallID := tcs.Get("0.id").String()
nameChunk := tcs.Get("0.function.name").String()
if nameChunk != "" {
st.FuncNames[idx] = nameChunk
}
existingCallID := st.FuncCallIDs[idx]
effectiveCallID := existingCallID
shouldEmitItem := false
if existingCallID == "" && newCallID != "" {
// First time seeing a valid call_id for this index
effectiveCallID = newCallID
st.FuncCallIDs[idx] = newCallID
shouldEmitItem = true
}
if shouldEmitItem && effectiveCallID != "" {
o := []byte(`{"type":"response.output_item.added","sequence_number":0,"output_index":0,"item":{"id":"","type":"function_call","status":"in_progress","arguments":"","call_id":"","name":""}}`)
o, _ = sjson.SetBytes(o, "sequence_number", nextSeq())
o, _ = sjson.SetBytes(o, "output_index", idx)
o, _ = sjson.SetBytes(o, "item.id", fmt.Sprintf("fc_%s", effectiveCallID))
o, _ = sjson.SetBytes(o, "item.call_id", effectiveCallID)
name := st.FuncNames[idx]
o, _ = sjson.SetBytes(o, "item.name", name)
out = append(out, emitRespEvent("response.output_item.added", o))
}
// Ensure args buffer exists for this index
if st.FuncArgsBuf[idx] == nil {
st.FuncArgsBuf[idx] = &strings.Builder{}
}
// Append arguments delta if available and we have a valid call_id to reference
if args := tcs.Get("0.function.arguments"); args.Exists() && args.String() != "" {
// Prefer an already known call_id; fall back to newCallID if first time
refCallID := st.FuncCallIDs[idx]
if refCallID == "" {
refCallID = newCallID
tcs.ForEach(func(_, tc gjson.Result) bool {
toolIndex := int(tc.Get("index").Int())
key := toolStateKey(idx, toolIndex)
newCallID := tc.Get("id").String()
nameChunk := tc.Get("function.name").String()
if nameChunk != "" {
st.FuncNames[key] = nameChunk
}
if refCallID != "" {
ad := []byte(`{"type":"response.function_call_arguments.delta","sequence_number":0,"item_id":"","output_index":0,"delta":""}`)
ad, _ = sjson.SetBytes(ad, "sequence_number", nextSeq())
ad, _ = sjson.SetBytes(ad, "item_id", fmt.Sprintf("fc_%s", refCallID))
ad, _ = sjson.SetBytes(ad, "output_index", idx)
ad, _ = sjson.SetBytes(ad, "delta", args.String())
out = append(out, emitRespEvent("response.function_call_arguments.delta", ad))
existingCallID := st.FuncCallIDs[key]
effectiveCallID := existingCallID
shouldEmitItem := false
if existingCallID == "" && newCallID != "" {
effectiveCallID = newCallID
st.FuncCallIDs[key] = newCallID
st.FuncOutputIx[key] = allocOutputIndex()
shouldEmitItem = true
}
st.FuncArgsBuf[idx].WriteString(args.String())
}
if shouldEmitItem && effectiveCallID != "" {
outputIndex := st.FuncOutputIx[key]
o := []byte(`{"type":"response.output_item.added","sequence_number":0,"output_index":0,"item":{"id":"","type":"function_call","status":"in_progress","arguments":"","call_id":"","name":""}}`)
o, _ = sjson.SetBytes(o, "sequence_number", nextSeq())
o, _ = sjson.SetBytes(o, "output_index", outputIndex)
o, _ = sjson.SetBytes(o, "item.id", fmt.Sprintf("fc_%s", effectiveCallID))
o, _ = sjson.SetBytes(o, "item.call_id", effectiveCallID)
o, _ = sjson.SetBytes(o, "item.name", st.FuncNames[key])
out = append(out, emitRespEvent("response.output_item.added", o))
}
if st.FuncArgsBuf[key] == nil {
st.FuncArgsBuf[key] = &strings.Builder{}
}
if args := tc.Get("function.arguments"); args.Exists() && args.String() != "" {
refCallID := st.FuncCallIDs[key]
if refCallID == "" {
refCallID = newCallID
}
if refCallID != "" {
outputIndex := st.FuncOutputIx[key]
ad := []byte(`{"type":"response.function_call_arguments.delta","sequence_number":0,"item_id":"","output_index":0,"delta":""}`)
ad, _ = sjson.SetBytes(ad, "sequence_number", nextSeq())
ad, _ = sjson.SetBytes(ad, "item_id", fmt.Sprintf("fc_%s", refCallID))
ad, _ = sjson.SetBytes(ad, "output_index", outputIndex)
ad, _ = sjson.SetBytes(ad, "delta", args.String())
out = append(out, emitRespEvent("response.function_call_arguments.delta", ad))
}
st.FuncArgsBuf[key].WriteString(args.String())
}
return true
})
}
}
@@ -360,15 +384,10 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
for i := range st.MsgItemAdded {
idxs = append(idxs, i)
}
for i := 0; i < len(idxs); i++ {
for j := i + 1; j < len(idxs); j++ {
if idxs[j] < idxs[i] {
idxs[i], idxs[j] = idxs[j], idxs[i]
}
}
}
sort.Slice(idxs, func(i, j int) bool { return st.MsgOutputIx[idxs[i]] < st.MsgOutputIx[idxs[j]] })
for _, i := range idxs {
if st.MsgItemAdded[i] && !st.MsgItemDone[i] {
msgOutputIndex := st.MsgOutputIx[i]
fullText := ""
if b := st.MsgTextBuf[i]; b != nil {
fullText = b.String()
@@ -376,7 +395,7 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
done := []byte(`{"type":"response.output_text.done","sequence_number":0,"item_id":"","output_index":0,"content_index":0,"text":"","logprobs":[]}`)
done, _ = sjson.SetBytes(done, "sequence_number", nextSeq())
done, _ = sjson.SetBytes(done, "item_id", fmt.Sprintf("msg_%s_%d", st.ResponseID, i))
done, _ = sjson.SetBytes(done, "output_index", i)
done, _ = sjson.SetBytes(done, "output_index", msgOutputIndex)
done, _ = sjson.SetBytes(done, "content_index", 0)
done, _ = sjson.SetBytes(done, "text", fullText)
out = append(out, emitRespEvent("response.output_text.done", done))
@@ -384,14 +403,14 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
partDone := []byte(`{"type":"response.content_part.done","sequence_number":0,"item_id":"","output_index":0,"content_index":0,"part":{"type":"output_text","annotations":[],"logprobs":[],"text":""}}`)
partDone, _ = sjson.SetBytes(partDone, "sequence_number", nextSeq())
partDone, _ = sjson.SetBytes(partDone, "item_id", fmt.Sprintf("msg_%s_%d", st.ResponseID, i))
partDone, _ = sjson.SetBytes(partDone, "output_index", i)
partDone, _ = sjson.SetBytes(partDone, "output_index", msgOutputIndex)
partDone, _ = sjson.SetBytes(partDone, "content_index", 0)
partDone, _ = sjson.SetBytes(partDone, "part.text", fullText)
out = append(out, emitRespEvent("response.content_part.done", partDone))
itemDone := []byte(`{"type":"response.output_item.done","sequence_number":0,"output_index":0,"item":{"id":"","type":"message","status":"completed","content":[{"type":"output_text","annotations":[],"logprobs":[],"text":""}],"role":"assistant"}}`)
itemDone, _ = sjson.SetBytes(itemDone, "sequence_number", nextSeq())
itemDone, _ = sjson.SetBytes(itemDone, "output_index", i)
itemDone, _ = sjson.SetBytes(itemDone, "output_index", msgOutputIndex)
itemDone, _ = sjson.SetBytes(itemDone, "item.id", fmt.Sprintf("msg_%s_%d", st.ResponseID, i))
itemDone, _ = sjson.SetBytes(itemDone, "item.content.0.text", fullText)
out = append(out, emitRespEvent("response.output_item.done", itemDone))
@@ -407,43 +426,42 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
// Emit function call done events for any active function calls
if len(st.FuncCallIDs) > 0 {
idxs := make([]int, 0, len(st.FuncCallIDs))
for i := range st.FuncCallIDs {
idxs = append(idxs, i)
keys := make([]string, 0, len(st.FuncCallIDs))
for key := range st.FuncCallIDs {
keys = append(keys, key)
}
for i := 0; i < len(idxs); i++ {
for j := i + 1; j < len(idxs); j++ {
if idxs[j] < idxs[i] {
idxs[i], idxs[j] = idxs[j], idxs[i]
}
}
}
for _, i := range idxs {
callID := st.FuncCallIDs[i]
if callID == "" || st.FuncItemDone[i] {
sort.Slice(keys, func(i, j int) bool {
left := st.FuncOutputIx[keys[i]]
right := st.FuncOutputIx[keys[j]]
return left < right || (left == right && keys[i] < keys[j])
})
for _, key := range keys {
callID := st.FuncCallIDs[key]
if callID == "" || st.FuncItemDone[key] {
continue
}
outputIndex := st.FuncOutputIx[key]
args := "{}"
if b := st.FuncArgsBuf[i]; b != nil && b.Len() > 0 {
if b := st.FuncArgsBuf[key]; b != nil && b.Len() > 0 {
args = b.String()
}
fcDone := []byte(`{"type":"response.function_call_arguments.done","sequence_number":0,"item_id":"","output_index":0,"arguments":""}`)
fcDone, _ = sjson.SetBytes(fcDone, "sequence_number", nextSeq())
fcDone, _ = sjson.SetBytes(fcDone, "item_id", fmt.Sprintf("fc_%s", callID))
fcDone, _ = sjson.SetBytes(fcDone, "output_index", i)
fcDone, _ = sjson.SetBytes(fcDone, "output_index", outputIndex)
fcDone, _ = sjson.SetBytes(fcDone, "arguments", args)
out = append(out, emitRespEvent("response.function_call_arguments.done", fcDone))
itemDone := []byte(`{"type":"response.output_item.done","sequence_number":0,"output_index":0,"item":{"id":"","type":"function_call","status":"completed","arguments":"","call_id":"","name":""}}`)
itemDone, _ = sjson.SetBytes(itemDone, "sequence_number", nextSeq())
itemDone, _ = sjson.SetBytes(itemDone, "output_index", i)
itemDone, _ = sjson.SetBytes(itemDone, "output_index", outputIndex)
itemDone, _ = sjson.SetBytes(itemDone, "item.id", fmt.Sprintf("fc_%s", callID))
itemDone, _ = sjson.SetBytes(itemDone, "item.arguments", args)
itemDone, _ = sjson.SetBytes(itemDone, "item.call_id", callID)
itemDone, _ = sjson.SetBytes(itemDone, "item.name", st.FuncNames[i])
itemDone, _ = sjson.SetBytes(itemDone, "item.name", st.FuncNames[key])
out = append(out, emitRespEvent("response.output_item.done", itemDone))
st.FuncItemDone[i] = true
st.FuncArgsDone[i] = true
st.FuncItemDone[key] = true
st.FuncArgsDone[key] = true
}
}
completed := []byte(`{"type":"response.completed","sequence_number":0,"response":{"id":"","object":"response","created_at":0,"status":"completed","background":false,"error":null}}`)
@@ -516,28 +534,21 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
}
// Build response.output using aggregated buffers
outputsWrapper := []byte(`{"arr":[]}`)
type completedOutputItem struct {
index int
raw []byte
}
outputItems := make([]completedOutputItem, 0, len(st.Reasonings)+len(st.MsgItemAdded)+len(st.FuncArgsBuf))
if len(st.Reasonings) > 0 {
for _, r := range st.Reasonings {
item := []byte(`{"id":"","type":"reasoning","summary":[{"type":"summary_text","text":""}]}`)
item, _ = sjson.SetBytes(item, "id", r.ReasoningID)
item, _ = sjson.SetBytes(item, "summary.0.text", r.ReasoningData)
outputsWrapper, _ = sjson.SetRawBytes(outputsWrapper, "arr.-1", item)
outputItems = append(outputItems, completedOutputItem{index: r.OutputIndex, raw: item})
}
}
// Append message items in ascending index order
if len(st.MsgItemAdded) > 0 {
midxs := make([]int, 0, len(st.MsgItemAdded))
for i := range st.MsgItemAdded {
midxs = append(midxs, i)
}
for i := 0; i < len(midxs); i++ {
for j := i + 1; j < len(midxs); j++ {
if midxs[j] < midxs[i] {
midxs[i], midxs[j] = midxs[j], midxs[i]
}
}
}
for _, i := range midxs {
txt := ""
if b := st.MsgTextBuf[i]; b != nil {
txt = b.String()
@@ -545,37 +556,29 @@ func ConvertOpenAIChatCompletionsResponseToOpenAIResponses(ctx context.Context,
item := []byte(`{"id":"","type":"message","status":"completed","content":[{"type":"output_text","annotations":[],"logprobs":[],"text":""}],"role":"assistant"}`)
item, _ = sjson.SetBytes(item, "id", fmt.Sprintf("msg_%s_%d", st.ResponseID, i))
item, _ = sjson.SetBytes(item, "content.0.text", txt)
outputsWrapper, _ = sjson.SetRawBytes(outputsWrapper, "arr.-1", item)
outputItems = append(outputItems, completedOutputItem{index: st.MsgOutputIx[i], raw: item})
}
}
if len(st.FuncArgsBuf) > 0 {
idxs := make([]int, 0, len(st.FuncArgsBuf))
for i := range st.FuncArgsBuf {
idxs = append(idxs, i)
}
// small-N sort without extra imports
for i := 0; i < len(idxs); i++ {
for j := i + 1; j < len(idxs); j++ {
if idxs[j] < idxs[i] {
idxs[i], idxs[j] = idxs[j], idxs[i]
}
}
}
for _, i := range idxs {
for key := range st.FuncArgsBuf {
args := ""
if b := st.FuncArgsBuf[i]; b != nil {
if b := st.FuncArgsBuf[key]; b != nil {
args = b.String()
}
callID := st.FuncCallIDs[i]
name := st.FuncNames[i]
callID := st.FuncCallIDs[key]
name := st.FuncNames[key]
item := []byte(`{"id":"","type":"function_call","status":"completed","arguments":"","call_id":"","name":""}`)
item, _ = sjson.SetBytes(item, "id", fmt.Sprintf("fc_%s", callID))
item, _ = sjson.SetBytes(item, "arguments", args)
item, _ = sjson.SetBytes(item, "call_id", callID)
item, _ = sjson.SetBytes(item, "name", name)
outputsWrapper, _ = sjson.SetRawBytes(outputsWrapper, "arr.-1", item)
outputItems = append(outputItems, completedOutputItem{index: st.FuncOutputIx[key], raw: item})
}
}
sort.Slice(outputItems, func(i, j int) bool { return outputItems[i].index < outputItems[j].index })
for _, item := range outputItems {
outputsWrapper, _ = sjson.SetRawBytes(outputsWrapper, "arr.-1", item.raw)
}
if gjson.GetBytes(outputsWrapper, "arr.#").Int() > 0 {
completed, _ = sjson.SetRawBytes(completed, "response.output", []byte(gjson.GetBytes(outputsWrapper, "arr").Raw))
}
@@ -0,0 +1,305 @@
package responses
import (
"context"
"strings"
"testing"
"github.com/tidwall/gjson"
)
func parseOpenAIResponsesSSEEvent(t *testing.T, chunk []byte) (string, gjson.Result) {
t.Helper()
lines := strings.Split(string(chunk), "\n")
if len(lines) < 2 {
t.Fatalf("unexpected SSE chunk: %q", chunk)
}
event := strings.TrimSpace(strings.TrimPrefix(lines[0], "event:"))
dataLine := strings.TrimSpace(strings.TrimPrefix(lines[1], "data:"))
if !gjson.Valid(dataLine) {
t.Fatalf("invalid SSE data JSON: %q", dataLine)
}
return event, gjson.Parse(dataLine)
}
func TestConvertOpenAIChatCompletionsResponseToOpenAIResponses_MultipleToolCallsRemainSeparate(t *testing.T) {
in := []string{
`data: {"id":"resp_test","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":[{"index":0,"id":"call_read","type":"function","function":{"name":"read","arguments":""}}]},"finish_reason":null}]}`,
`data: {"id":"resp_test","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"index":0,"function":{"arguments":"{\"filePath\":\"C:\\\\repo\",\"limit\":400,\"offset\":1}"}}]},"finish_reason":null}]}`,
`data: {"id":"resp_test","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":[{"index":1,"id":"call_glob","type":"function","function":{"name":"glob","arguments":""}}]},"finish_reason":null}]}`,
`data: {"id":"resp_test","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"index":1,"function":{"arguments":"{\"path\":\"C:\\\\repo\",\"pattern\":\"*.{yml,yaml}\"}"}}]},"finish_reason":null}]}`,
`data: {"id":"resp_test","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"finish_reason":"tool_calls"}],"usage":{"completion_tokens":10,"total_tokens":20,"prompt_tokens":10}}`,
}
request := []byte(`{"model":"gpt-5.4","tool_choice":"auto","parallel_tool_calls":true}`)
var param any
var out [][]byte
for _, line := range in {
out = append(out, ConvertOpenAIChatCompletionsResponseToOpenAIResponses(context.Background(), "model", request, request, []byte(line), &param)...)
}
addedNames := map[string]string{}
doneArgs := map[string]string{}
doneNames := map[string]string{}
outputItems := map[string]gjson.Result{}
for _, chunk := range out {
ev, data := parseOpenAIResponsesSSEEvent(t, chunk)
switch ev {
case "response.output_item.added":
if data.Get("item.type").String() != "function_call" {
continue
}
addedNames[data.Get("item.call_id").String()] = data.Get("item.name").String()
case "response.output_item.done":
if data.Get("item.type").String() != "function_call" {
continue
}
callID := data.Get("item.call_id").String()
doneArgs[callID] = data.Get("item.arguments").String()
doneNames[callID] = data.Get("item.name").String()
case "response.completed":
output := data.Get("response.output")
for _, item := range output.Array() {
if item.Get("type").String() == "function_call" {
outputItems[item.Get("call_id").String()] = item
}
}
}
}
if len(addedNames) != 2 {
t.Fatalf("expected 2 function_call added events, got %d", len(addedNames))
}
if len(doneArgs) != 2 {
t.Fatalf("expected 2 function_call done events, got %d", len(doneArgs))
}
if addedNames["call_read"] != "read" {
t.Fatalf("unexpected added name for call_read: %q", addedNames["call_read"])
}
if addedNames["call_glob"] != "glob" {
t.Fatalf("unexpected added name for call_glob: %q", addedNames["call_glob"])
}
if !gjson.Valid(doneArgs["call_read"]) {
t.Fatalf("invalid JSON args for call_read: %q", doneArgs["call_read"])
}
if !gjson.Valid(doneArgs["call_glob"]) {
t.Fatalf("invalid JSON args for call_glob: %q", doneArgs["call_glob"])
}
if strings.Contains(doneArgs["call_read"], "}{") {
t.Fatalf("call_read args were concatenated: %q", doneArgs["call_read"])
}
if strings.Contains(doneArgs["call_glob"], "}{") {
t.Fatalf("call_glob args were concatenated: %q", doneArgs["call_glob"])
}
if doneNames["call_read"] != "read" {
t.Fatalf("unexpected done name for call_read: %q", doneNames["call_read"])
}
if doneNames["call_glob"] != "glob" {
t.Fatalf("unexpected done name for call_glob: %q", doneNames["call_glob"])
}
if got := gjson.Get(doneArgs["call_read"], "filePath").String(); got != `C:\repo` {
t.Fatalf("unexpected filePath for call_read: %q", got)
}
if got := gjson.Get(doneArgs["call_glob"], "path").String(); got != `C:\repo` {
t.Fatalf("unexpected path for call_glob: %q", got)
}
if got := gjson.Get(doneArgs["call_glob"], "pattern").String(); got != "*.{yml,yaml}" {
t.Fatalf("unexpected pattern for call_glob: %q", got)
}
if len(outputItems) != 2 {
t.Fatalf("expected 2 function_call items in response.output, got %d", len(outputItems))
}
if outputItems["call_read"].Get("name").String() != "read" {
t.Fatalf("unexpected response.output name for call_read: %q", outputItems["call_read"].Get("name").String())
}
if outputItems["call_glob"].Get("name").String() != "glob" {
t.Fatalf("unexpected response.output name for call_glob: %q", outputItems["call_glob"].Get("name").String())
}
}
func TestConvertOpenAIChatCompletionsResponseToOpenAIResponses_MultiChoiceToolCallsUseDistinctOutputIndexes(t *testing.T) {
in := []string{
`data: {"id":"resp_multi_choice","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":[{"index":0,"id":"call_choice0","type":"function","function":{"name":"glob","arguments":""}}]},"finish_reason":null},{"index":1,"delta":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":[{"index":0,"id":"call_choice1","type":"function","function":{"name":"read","arguments":""}}]},"finish_reason":null}]}`,
`data: {"id":"resp_multi_choice","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"index":0,"function":{"arguments":"{\"path\":\"C:\\\\repo\",\"pattern\":\"*.go\"}"}}]},"finish_reason":null},{"index":1,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"index":0,"function":{"arguments":"{\"filePath\":\"C:\\\\repo\\\\README.md\",\"limit\":20,\"offset\":1}"}}]},"finish_reason":null}]}`,
`data: {"id":"resp_multi_choice","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"finish_reason":"tool_calls"},{"index":1,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"finish_reason":"tool_calls"}],"usage":{"completion_tokens":10,"total_tokens":20,"prompt_tokens":10}}`,
}
request := []byte(`{"model":"gpt-5.4","tool_choice":"auto","parallel_tool_calls":true}`)
var param any
var out [][]byte
for _, line := range in {
out = append(out, ConvertOpenAIChatCompletionsResponseToOpenAIResponses(context.Background(), "model", request, request, []byte(line), &param)...)
}
type fcEvent struct {
outputIndex int64
name string
arguments string
}
added := map[string]fcEvent{}
done := map[string]fcEvent{}
for _, chunk := range out {
ev, data := parseOpenAIResponsesSSEEvent(t, chunk)
switch ev {
case "response.output_item.added":
if data.Get("item.type").String() != "function_call" {
continue
}
callID := data.Get("item.call_id").String()
added[callID] = fcEvent{
outputIndex: data.Get("output_index").Int(),
name: data.Get("item.name").String(),
}
case "response.output_item.done":
if data.Get("item.type").String() != "function_call" {
continue
}
callID := data.Get("item.call_id").String()
done[callID] = fcEvent{
outputIndex: data.Get("output_index").Int(),
name: data.Get("item.name").String(),
arguments: data.Get("item.arguments").String(),
}
}
}
if len(added) != 2 {
t.Fatalf("expected 2 function_call added events, got %d", len(added))
}
if len(done) != 2 {
t.Fatalf("expected 2 function_call done events, got %d", len(done))
}
if added["call_choice0"].name != "glob" {
t.Fatalf("unexpected added name for call_choice0: %q", added["call_choice0"].name)
}
if added["call_choice1"].name != "read" {
t.Fatalf("unexpected added name for call_choice1: %q", added["call_choice1"].name)
}
if added["call_choice0"].outputIndex == added["call_choice1"].outputIndex {
t.Fatalf("expected distinct output indexes for different choices, both got %d", added["call_choice0"].outputIndex)
}
if !gjson.Valid(done["call_choice0"].arguments) {
t.Fatalf("invalid JSON args for call_choice0: %q", done["call_choice0"].arguments)
}
if !gjson.Valid(done["call_choice1"].arguments) {
t.Fatalf("invalid JSON args for call_choice1: %q", done["call_choice1"].arguments)
}
if done["call_choice0"].outputIndex == done["call_choice1"].outputIndex {
t.Fatalf("expected distinct done output indexes for different choices, both got %d", done["call_choice0"].outputIndex)
}
if done["call_choice0"].name != "glob" {
t.Fatalf("unexpected done name for call_choice0: %q", done["call_choice0"].name)
}
if done["call_choice1"].name != "read" {
t.Fatalf("unexpected done name for call_choice1: %q", done["call_choice1"].name)
}
}
func TestConvertOpenAIChatCompletionsResponseToOpenAIResponses_MixedMessageAndToolUseDistinctOutputIndexes(t *testing.T) {
in := []string{
`data: {"id":"resp_mixed","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":"assistant","content":"hello","reasoning_content":null,"tool_calls":null},"finish_reason":null},{"index":1,"delta":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":[{"index":0,"id":"call_choice1","type":"function","function":{"name":"read","arguments":""}}]},"finish_reason":null}]}`,
`data: {"id":"resp_mixed","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"finish_reason":"stop"},{"index":1,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"index":0,"function":{"arguments":"{\"filePath\":\"C:\\\\repo\\\\README.md\",\"limit\":20,\"offset\":1}"}}]},"finish_reason":"tool_calls"}],"usage":{"completion_tokens":10,"total_tokens":20,"prompt_tokens":10}}`,
}
request := []byte(`{"model":"gpt-5.4","tool_choice":"auto","parallel_tool_calls":true}`)
var param any
var out [][]byte
for _, line := range in {
out = append(out, ConvertOpenAIChatCompletionsResponseToOpenAIResponses(context.Background(), "model", request, request, []byte(line), &param)...)
}
var messageOutputIndex int64 = -1
var toolOutputIndex int64 = -1
for _, chunk := range out {
ev, data := parseOpenAIResponsesSSEEvent(t, chunk)
if ev != "response.output_item.added" {
continue
}
switch data.Get("item.type").String() {
case "message":
if data.Get("item.id").String() == "msg_resp_mixed_0" {
messageOutputIndex = data.Get("output_index").Int()
}
case "function_call":
if data.Get("item.call_id").String() == "call_choice1" {
toolOutputIndex = data.Get("output_index").Int()
}
}
}
if messageOutputIndex < 0 {
t.Fatal("did not find message output index")
}
if toolOutputIndex < 0 {
t.Fatal("did not find tool output index")
}
if messageOutputIndex == toolOutputIndex {
t.Fatalf("expected distinct output indexes for message and tool call, both got %d", messageOutputIndex)
}
}
func TestConvertOpenAIChatCompletionsResponseToOpenAIResponses_FunctionCallDoneAndCompletedOutputStayAscending(t *testing.T) {
in := []string{
`data: {"id":"resp_order","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":[{"index":0,"id":"call_glob","type":"function","function":{"name":"glob","arguments":""}}]},"finish_reason":null}]}`,
`data: {"id":"resp_order","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"index":0,"function":{"arguments":"{\"path\":\"C:\\\\repo\",\"pattern\":\"*.go\"}"}}]},"finish_reason":null}]}`,
`data: {"id":"resp_order","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":"assistant","content":null,"reasoning_content":null,"tool_calls":[{"index":1,"id":"call_read","type":"function","function":{"name":"read","arguments":""}}]},"finish_reason":null}]}`,
`data: {"id":"resp_order","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"index":1,"function":{"arguments":"{\"filePath\":\"C:\\\\repo\\\\README.md\",\"limit\":20,\"offset\":1}"}}]},"finish_reason":null}]}`,
`data: {"id":"resp_order","object":"chat.completion.chunk","created":1773896263,"model":"model","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"finish_reason":"tool_calls"}],"usage":{"completion_tokens":10,"total_tokens":20,"prompt_tokens":10}}`,
}
request := []byte(`{"model":"gpt-5.4","tool_choice":"auto","parallel_tool_calls":true}`)
var param any
var out [][]byte
for _, line := range in {
out = append(out, ConvertOpenAIChatCompletionsResponseToOpenAIResponses(context.Background(), "model", request, request, []byte(line), &param)...)
}
var doneIndexes []int64
var completedOrder []string
for _, chunk := range out {
ev, data := parseOpenAIResponsesSSEEvent(t, chunk)
switch ev {
case "response.output_item.done":
if data.Get("item.type").String() == "function_call" {
doneIndexes = append(doneIndexes, data.Get("output_index").Int())
}
case "response.completed":
for _, item := range data.Get("response.output").Array() {
if item.Get("type").String() == "function_call" {
completedOrder = append(completedOrder, item.Get("call_id").String())
}
}
}
}
if len(doneIndexes) != 2 {
t.Fatalf("expected 2 function_call done indexes, got %d", len(doneIndexes))
}
if doneIndexes[0] >= doneIndexes[1] {
t.Fatalf("expected ascending done output indexes, got %v", doneIndexes)
}
if len(completedOrder) != 2 {
t.Fatalf("expected 2 function_call items in completed output, got %d", len(completedOrder))
}
if completedOrder[0] != "call_glob" || completedOrder[1] != "call_read" {
t.Fatalf("unexpected completed function_call order: %v", completedOrder)
}
}
+13 -1
View File
@@ -87,9 +87,10 @@ type modelStats struct {
Details []RequestDetail
}
// RequestDetail stores the timestamp and token usage for a single request.
// RequestDetail stores the timestamp, latency, and token usage for a single request.
type RequestDetail struct {
Timestamp time.Time `json:"timestamp"`
LatencyMs int64 `json:"latency_ms"`
Source string `json:"source"`
AuthIndex string `json:"auth_index"`
Tokens TokenStats `json:"tokens"`
@@ -198,6 +199,7 @@ func (s *RequestStatistics) Record(ctx context.Context, record coreusage.Record)
}
s.updateAPIStats(stats, modelName, RequestDetail{
Timestamp: timestamp,
LatencyMs: normaliseLatency(record.Latency),
Source: record.Source,
AuthIndex: record.AuthIndex,
Tokens: detail,
@@ -332,6 +334,9 @@ func (s *RequestStatistics) MergeSnapshot(snapshot StatisticsSnapshot) MergeResu
}
for _, detail := range modelSnapshot.Details {
detail.Tokens = normaliseTokenStats(detail.Tokens)
if detail.LatencyMs < 0 {
detail.LatencyMs = 0
}
if detail.Timestamp.IsZero() {
detail.Timestamp = time.Now()
}
@@ -463,6 +468,13 @@ func normaliseTokenStats(tokens TokenStats) TokenStats {
return tokens
}
func normaliseLatency(latency time.Duration) int64 {
if latency <= 0 {
return 0
}
return latency.Milliseconds()
}
func formatHour(hour int) string {
if hour < 0 {
hour = 0
+96
View File
@@ -0,0 +1,96 @@
package usage
import (
"context"
"testing"
"time"
coreusage "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/usage"
)
func TestRequestStatisticsRecordIncludesLatency(t *testing.T) {
stats := NewRequestStatistics()
stats.Record(context.Background(), coreusage.Record{
APIKey: "test-key",
Model: "gpt-5.4",
RequestedAt: time.Date(2026, 3, 20, 12, 0, 0, 0, time.UTC),
Latency: 1500 * time.Millisecond,
Detail: coreusage.Detail{
InputTokens: 10,
OutputTokens: 20,
TotalTokens: 30,
},
})
snapshot := stats.Snapshot()
details := snapshot.APIs["test-key"].Models["gpt-5.4"].Details
if len(details) != 1 {
t.Fatalf("details len = %d, want 1", len(details))
}
if details[0].LatencyMs != 1500 {
t.Fatalf("latency_ms = %d, want 1500", details[0].LatencyMs)
}
}
func TestRequestStatisticsMergeSnapshotDedupIgnoresLatency(t *testing.T) {
stats := NewRequestStatistics()
timestamp := time.Date(2026, 3, 20, 12, 0, 0, 0, time.UTC)
first := StatisticsSnapshot{
APIs: map[string]APISnapshot{
"test-key": {
Models: map[string]ModelSnapshot{
"gpt-5.4": {
Details: []RequestDetail{{
Timestamp: timestamp,
LatencyMs: 0,
Source: "user@example.com",
AuthIndex: "0",
Tokens: TokenStats{
InputTokens: 10,
OutputTokens: 20,
TotalTokens: 30,
},
}},
},
},
},
},
}
second := StatisticsSnapshot{
APIs: map[string]APISnapshot{
"test-key": {
Models: map[string]ModelSnapshot{
"gpt-5.4": {
Details: []RequestDetail{{
Timestamp: timestamp,
LatencyMs: 2500,
Source: "user@example.com",
AuthIndex: "0",
Tokens: TokenStats{
InputTokens: 10,
OutputTokens: 20,
TotalTokens: 30,
},
}},
},
},
},
},
}
result := stats.MergeSnapshot(first)
if result.Added != 1 || result.Skipped != 0 {
t.Fatalf("first merge = %+v, want added=1 skipped=0", result)
}
result = stats.MergeSnapshot(second)
if result.Added != 0 || result.Skipped != 1 {
t.Fatalf("second merge = %+v, want added=0 skipped=1", result)
}
snapshot := stats.Snapshot()
details := snapshot.APIs["test-key"].Models["gpt-5.4"].Details
if len(details) != 1 {
t.Fatalf("details len = %d, want 1", len(details))
}
}
+74
View File
@@ -54,3 +54,77 @@ func TestSanitizeFunctionName(t *testing.T) {
})
}
}
func TestSanitizedToolNameMap(t *testing.T) {
t.Run("returns map for tools needing sanitization", func(t *testing.T) {
raw := []byte(`{"tools":[
{"name":"valid_tool","input_schema":{}},
{"name":"mcp/server/read","input_schema":{}},
{"name":"tool@v2","input_schema":{}}
]}`)
m := SanitizedToolNameMap(raw)
if m == nil {
t.Fatal("expected non-nil map")
}
if m["mcp_server_read"] != "mcp/server/read" {
t.Errorf("expected mcp_server_read → mcp/server/read, got %q", m["mcp_server_read"])
}
if m["tool_v2"] != "tool@v2" {
t.Errorf("expected tool_v2 → tool@v2, got %q", m["tool_v2"])
}
if _, exists := m["valid_tool"]; exists {
t.Error("valid_tool should not be in the map (no sanitization needed)")
}
})
t.Run("returns nil when no tools need sanitization", func(t *testing.T) {
raw := []byte(`{"tools":[{"name":"Read","input_schema":{}},{"name":"Write","input_schema":{}}]}`)
m := SanitizedToolNameMap(raw)
if m != nil {
t.Errorf("expected nil, got %v", m)
}
})
t.Run("returns nil for empty/missing tools", func(t *testing.T) {
if m := SanitizedToolNameMap([]byte(`{}`)); m != nil {
t.Error("expected nil for no tools")
}
if m := SanitizedToolNameMap(nil); m != nil {
t.Error("expected nil for nil input")
}
})
t.Run("collision keeps first mapping", func(t *testing.T) {
raw := []byte(`{"tools":[
{"name":"read/file","input_schema":{}},
{"name":"read@file","input_schema":{}}
]}`)
m := SanitizedToolNameMap(raw)
if m == nil {
t.Fatal("expected non-nil map")
}
if m["read_file"] != "read/file" {
t.Errorf("expected first mapping read/file, got %q", m["read_file"])
}
})
}
func TestRestoreSanitizedToolName(t *testing.T) {
m := map[string]string{
"mcp_server_read": "mcp/server/read",
"tool_v2": "tool@v2",
}
if got := RestoreSanitizedToolName(m, "mcp_server_read"); got != "mcp/server/read" {
t.Errorf("expected mcp/server/read, got %q", got)
}
if got := RestoreSanitizedToolName(m, "unknown"); got != "unknown" {
t.Errorf("expected passthrough for unknown, got %q", got)
}
if got := RestoreSanitizedToolName(nil, "name"); got != "name" {
t.Errorf("expected passthrough for nil map, got %q", got)
}
if got := RestoreSanitizedToolName(m, ""); got != "" {
t.Errorf("expected empty for empty name, got %q", got)
}
}
+55
View File
@@ -8,6 +8,7 @@ import (
"fmt"
"strings"
log "github.com/sirupsen/logrus"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
)
@@ -243,6 +244,9 @@ func ToolNameMapFromClaudeRequest(rawJSON []byte) map[string]string {
out := make(map[string]string, len(toolResults))
tools.ForEach(func(_, tool gjson.Result) bool {
name := strings.TrimSpace(tool.Get("name").String())
if name == "" {
name = strings.TrimSpace(tool.Get("function.name").String())
}
if name == "" {
return true
}
@@ -271,3 +275,54 @@ func MapToolName(toolNameMap map[string]string, name string) string {
}
return name
}
// SanitizedToolNameMap builds a sanitized-name → original-name map from Claude request tools.
// It is used to restore exact tool names for clients (e.g. Claude Code) after the proxy
// sanitizes tool names for Gemini/Vertex API compatibility via SanitizeFunctionName.
// Only entries where sanitization actually changes the name are included.
func SanitizedToolNameMap(rawJSON []byte) map[string]string {
if len(rawJSON) == 0 || !gjson.ValidBytes(rawJSON) {
return nil
}
tools := gjson.GetBytes(rawJSON, "tools")
if !tools.Exists() || !tools.IsArray() {
return nil
}
out := make(map[string]string)
tools.ForEach(func(_, tool gjson.Result) bool {
name := strings.TrimSpace(tool.Get("name").String())
if name == "" {
return true
}
sanitized := SanitizeFunctionName(name)
if sanitized == name {
return true
}
if _, exists := out[sanitized]; !exists {
out[sanitized] = name
} else {
log.Warnf("sanitized tool name collision: %q and %q both map to %q, keeping first", out[sanitized], name, sanitized)
}
return true
})
if len(out) == 0 {
return nil
}
return out
}
// RestoreSanitizedToolName looks up a sanitized function name in the provided map
// and returns the original client-facing name. If no mapping exists, it returns
// the sanitized name unchanged.
func RestoreSanitizedToolName(toolNameMap map[string]string, sanitizedName string) string {
if sanitizedName == "" || toolNameMap == nil {
return sanitizedName
}
if original, ok := toolNameMap[sanitizedName]; ok {
return original
}
return sanitizedName
}
+35 -15
View File
@@ -75,7 +75,12 @@ func (w *Watcher) reloadClients(rescanAuth bool, affectedOAuthProviders []string
w.clientsMutex.Lock()
w.lastAuthHashes = make(map[string]string)
w.lastAuthContents = make(map[string]*coreauth.Auth)
cacheAuthContents := log.IsLevelEnabled(log.DebugLevel)
if cacheAuthContents {
w.lastAuthContents = make(map[string]*coreauth.Auth)
} else {
w.lastAuthContents = nil
}
w.fileAuthsByPath = make(map[string]map[string]*coreauth.Auth)
if resolvedAuthDir, errResolveAuthDir := util.ResolveAuthDir(cfg.AuthDir); errResolveAuthDir != nil {
log.Errorf("failed to resolve auth directory for hash cache: %v", errResolveAuthDir)
@@ -89,10 +94,12 @@ func (w *Watcher) reloadClients(rescanAuth bool, affectedOAuthProviders []string
sum := sha256.Sum256(data)
normalizedPath := w.normalizeAuthPath(path)
w.lastAuthHashes[normalizedPath] = hex.EncodeToString(sum[:])
// Parse and cache auth content for future diff comparisons
var auth coreauth.Auth
if errParse := json.Unmarshal(data, &auth); errParse == nil {
w.lastAuthContents[normalizedPath] = &auth
// Parse and cache auth content for future diff comparisons (debug only).
if cacheAuthContents {
var auth coreauth.Auth
if errParse := json.Unmarshal(data, &auth); errParse == nil {
w.lastAuthContents[normalizedPath] = &auth
}
}
ctx := &synthesizer.SynthesisContext{
Config: cfg,
@@ -102,7 +109,7 @@ func (w *Watcher) reloadClients(rescanAuth bool, affectedOAuthProviders []string
}
if generated := synthesizer.SynthesizeAuthFile(ctx, path, data); len(generated) > 0 {
if pathAuths := authSliceToMap(generated); len(pathAuths) > 0 {
w.fileAuthsByPath[normalizedPath] = pathAuths
w.fileAuthsByPath[normalizedPath] = authIDSet(pathAuths)
}
}
}
@@ -171,25 +178,30 @@ func (w *Watcher) addOrUpdateClient(path string) {
}
// Get old auth for diff comparison
cacheAuthContents := log.IsLevelEnabled(log.DebugLevel)
var oldAuth *coreauth.Auth
if w.lastAuthContents != nil {
if cacheAuthContents && w.lastAuthContents != nil {
oldAuth = w.lastAuthContents[normalized]
}
// Compute and log field changes
if changes := diff.BuildAuthChangeDetails(oldAuth, &newAuth); len(changes) > 0 {
log.Debugf("auth field changes for %s:", filepath.Base(path))
for _, c := range changes {
log.Debugf(" %s", c)
if cacheAuthContents {
if changes := diff.BuildAuthChangeDetails(oldAuth, &newAuth); len(changes) > 0 {
log.Debugf("auth field changes for %s:", filepath.Base(path))
for _, c := range changes {
log.Debugf(" %s", c)
}
}
}
// Update caches
w.lastAuthHashes[normalized] = curHash
if w.lastAuthContents == nil {
w.lastAuthContents = make(map[string]*coreauth.Auth)
if cacheAuthContents {
if w.lastAuthContents == nil {
w.lastAuthContents = make(map[string]*coreauth.Auth)
}
w.lastAuthContents[normalized] = &newAuth
}
w.lastAuthContents[normalized] = &newAuth
oldByID := make(map[string]*coreauth.Auth, len(w.fileAuthsByPath[normalized]))
for id, a := range w.fileAuthsByPath[normalized] {
@@ -206,7 +218,7 @@ func (w *Watcher) addOrUpdateClient(path string) {
generated := synthesizer.SynthesizeAuthFile(sctx, path, data)
newByID := authSliceToMap(generated)
if len(newByID) > 0 {
w.fileAuthsByPath[normalized] = newByID
w.fileAuthsByPath[normalized] = authIDSet(newByID)
} else {
delete(w.fileAuthsByPath, normalized)
}
@@ -273,6 +285,14 @@ func authSliceToMap(auths []*coreauth.Auth) map[string]*coreauth.Auth {
return byID
}
func authIDSet(auths map[string]*coreauth.Auth) map[string]*coreauth.Auth {
set := make(map[string]*coreauth.Auth, len(auths))
for id := range auths {
set[id] = nil
}
return set
}
func (w *Watcher) loadFileClients(cfg *config.Config) int {
authFileCount := 0
successfulAuthCount := 0
+3
View File
@@ -256,6 +256,9 @@ func BuildConfigChangeDetails(oldCfg, newCfg *config.Config) []string {
if oldCfg.RemoteManagement.DisableControlPanel != newCfg.RemoteManagement.DisableControlPanel {
changes = append(changes, fmt.Sprintf("remote-management.disable-control-panel: %t -> %t", oldCfg.RemoteManagement.DisableControlPanel, newCfg.RemoteManagement.DisableControlPanel))
}
if oldCfg.RemoteManagement.DisableAutoUpdatePanel != newCfg.RemoteManagement.DisableAutoUpdatePanel {
changes = append(changes, fmt.Sprintf("remote-management.disable-auto-update-panel: %t -> %t", oldCfg.RemoteManagement.DisableAutoUpdatePanel, newCfg.RemoteManagement.DisableAutoUpdatePanel))
}
oldPanelRepo := strings.TrimSpace(oldCfg.RemoteManagement.PanelGitHubRepository)
newPanelRepo := strings.TrimSpace(newCfg.RemoteManagement.PanelGitHubRepository)
if oldPanelRepo != newPanelRepo {
+27 -19
View File
@@ -20,10 +20,11 @@ func TestBuildConfigChangeDetails(t *testing.T) {
RestrictManagementToLocalhost: false,
},
RemoteManagement: config.RemoteManagement{
AllowRemote: false,
SecretKey: "old",
DisableControlPanel: false,
PanelGitHubRepository: "repo-old",
AllowRemote: false,
SecretKey: "old",
DisableControlPanel: false,
DisableAutoUpdatePanel: false,
PanelGitHubRepository: "repo-old",
},
OAuthExcludedModels: map[string][]string{
"providerA": {"m1"},
@@ -54,10 +55,11 @@ func TestBuildConfigChangeDetails(t *testing.T) {
},
},
RemoteManagement: config.RemoteManagement{
AllowRemote: true,
SecretKey: "new",
DisableControlPanel: true,
PanelGitHubRepository: "repo-new",
AllowRemote: true,
SecretKey: "new",
DisableControlPanel: true,
DisableAutoUpdatePanel: true,
PanelGitHubRepository: "repo-new",
},
OAuthExcludedModels: map[string][]string{
"providerA": {"m1", "m2"},
@@ -88,6 +90,7 @@ func TestBuildConfigChangeDetails(t *testing.T) {
expectContains(t, details, "ampcode.upstream-url: http://old-upstream -> http://new-upstream")
expectContains(t, details, "ampcode.model-mappings: updated (1 -> 2 entries)")
expectContains(t, details, "remote-management.allow-remote: false -> true")
expectContains(t, details, "remote-management.disable-auto-update-panel: false -> true")
expectContains(t, details, "remote-management.secret-key: updated")
expectContains(t, details, "oauth-excluded-models[providera]: updated (1 -> 2 entries)")
expectContains(t, details, "oauth-excluded-models[providerb]: added (1 entries)")
@@ -265,9 +268,10 @@ func TestBuildConfigChangeDetails_FlagsAndKeys(t *testing.T) {
ModelMappings: []config.AmpModelMapping{{From: "a", To: "b"}},
},
RemoteManagement: config.RemoteManagement{
DisableControlPanel: true,
PanelGitHubRepository: "new/repo",
SecretKey: "",
DisableControlPanel: true,
DisableAutoUpdatePanel: true,
PanelGitHubRepository: "new/repo",
SecretKey: "",
},
SDKConfig: sdkconfig.SDKConfig{
RequestLog: true,
@@ -299,6 +303,7 @@ func TestBuildConfigChangeDetails_FlagsAndKeys(t *testing.T) {
expectContains(t, details, "ampcode.restrict-management-to-localhost: false -> true")
expectContains(t, details, "ampcode.upstream-api-key: removed")
expectContains(t, details, "remote-management.disable-control-panel: false -> true")
expectContains(t, details, "remote-management.disable-auto-update-panel: false -> true")
expectContains(t, details, "remote-management.panel-github-repository: old/repo -> new/repo")
expectContains(t, details, "remote-management.secret-key: deleted")
}
@@ -336,10 +341,11 @@ func TestBuildConfigChangeDetails_AllBranches(t *testing.T) {
ForceModelMappings: false,
},
RemoteManagement: config.RemoteManagement{
AllowRemote: false,
DisableControlPanel: false,
PanelGitHubRepository: "old/repo",
SecretKey: "old",
AllowRemote: false,
DisableControlPanel: false,
DisableAutoUpdatePanel: false,
PanelGitHubRepository: "old/repo",
SecretKey: "old",
},
SDKConfig: sdkconfig.SDKConfig{
RequestLog: false,
@@ -389,10 +395,11 @@ func TestBuildConfigChangeDetails_AllBranches(t *testing.T) {
ForceModelMappings: true,
},
RemoteManagement: config.RemoteManagement{
AllowRemote: true,
DisableControlPanel: true,
PanelGitHubRepository: "new/repo",
SecretKey: "",
AllowRemote: true,
DisableControlPanel: true,
DisableAutoUpdatePanel: true,
PanelGitHubRepository: "new/repo",
SecretKey: "",
},
SDKConfig: sdkconfig.SDKConfig{
RequestLog: true,
@@ -460,6 +467,7 @@ func TestBuildConfigChangeDetails_AllBranches(t *testing.T) {
expectContains(t, changes, "oauth-excluded-models[p2]: added (1 entries)")
expectContains(t, changes, "remote-management.allow-remote: false -> true")
expectContains(t, changes, "remote-management.disable-control-panel: false -> true")
expectContains(t, changes, "remote-management.disable-auto-update-panel: false -> true")
expectContains(t, changes, "remote-management.panel-github-repository: old/repo -> new/repo")
expectContains(t, changes, "remote-management.secret-key: deleted")
expectContains(t, changes, "openai-compatibility:")
+2 -1
View File
@@ -339,12 +339,13 @@ func (h *BaseAPIHandler) GetContextWithCancel(handler interfaces.APIHandler, c *
}
}
newCtx, cancel := context.WithCancel(parentCtx)
cancelCtx := newCtx
if requestCtx != nil && requestCtx != parentCtx {
go func() {
select {
case <-requestCtx.Done():
cancel()
case <-newCtx.Done():
case <-cancelCtx.Done():
}
}()
}
+298 -96
View File
@@ -421,10 +421,6 @@ func preserveRequestedModelSuffix(requestedModel, resolved string) string {
}
func (m *Manager) executionModelCandidates(auth *Auth, routeModel string) []string {
return m.prepareExecutionModels(auth, routeModel)
}
func (m *Manager) prepareExecutionModels(auth *Auth, routeModel string) []string {
requestedModel := rewriteModelForAuth(routeModel, auth)
requestedModel = m.applyOAuthModelAlias(auth, requestedModel)
if pool := m.resolveOpenAICompatUpstreamModelPool(auth, requestedModel); len(pool) > 0 {
@@ -441,6 +437,46 @@ func (m *Manager) prepareExecutionModels(auth *Auth, routeModel string) []string
return []string{resolved}
}
func executionResultModel(routeModel, upstreamModel string, pooled bool) string {
if pooled {
if resolved := strings.TrimSpace(upstreamModel); resolved != "" {
return resolved
}
}
if requested := strings.TrimSpace(routeModel); requested != "" {
return requested
}
return strings.TrimSpace(upstreamModel)
}
func filterExecutionModels(auth *Auth, routeModel string, candidates []string, pooled bool) []string {
if len(candidates) == 0 {
return nil
}
now := time.Now()
out := make([]string, 0, len(candidates))
for _, upstreamModel := range candidates {
stateModel := executionResultModel(routeModel, upstreamModel, pooled)
blocked, _, _ := isAuthBlockedForModel(auth, stateModel, now)
if blocked {
continue
}
out = append(out, upstreamModel)
}
return out
}
func (m *Manager) preparedExecutionModels(auth *Auth, routeModel string) ([]string, bool) {
candidates := m.executionModelCandidates(auth, routeModel)
pooled := len(candidates) > 1
return filterExecutionModels(auth, routeModel, candidates, pooled), pooled
}
func (m *Manager) prepareExecutionModels(auth *Auth, routeModel string) []string {
models, _ := m.preparedExecutionModels(auth, routeModel)
return models
}
func discardStreamChunks(ch <-chan cliproxyexecutor.StreamChunk) {
if ch == nil {
return
@@ -451,6 +487,59 @@ func discardStreamChunks(ch <-chan cliproxyexecutor.StreamChunk) {
}()
}
type streamBootstrapError struct {
cause error
headers http.Header
}
func cloneHTTPHeader(headers http.Header) http.Header {
if headers == nil {
return nil
}
return headers.Clone()
}
func newStreamBootstrapError(err error, headers http.Header) error {
if err == nil {
return nil
}
return &streamBootstrapError{
cause: err,
headers: cloneHTTPHeader(headers),
}
}
func (e *streamBootstrapError) Error() string {
if e == nil || e.cause == nil {
return ""
}
return e.cause.Error()
}
func (e *streamBootstrapError) Unwrap() error {
if e == nil {
return nil
}
return e.cause
}
func (e *streamBootstrapError) Headers() http.Header {
if e == nil {
return nil
}
return cloneHTTPHeader(e.headers)
}
func streamErrorResult(headers http.Header, err error) *cliproxyexecutor.StreamResult {
ch := make(chan cliproxyexecutor.StreamChunk, 1)
ch <- cliproxyexecutor.StreamChunk{Err: err}
close(ch)
return &cliproxyexecutor.StreamResult{
Headers: cloneHTTPHeader(headers),
Chunks: ch,
}
}
func readStreamBootstrap(ctx context.Context, ch <-chan cliproxyexecutor.StreamChunk) ([]cliproxyexecutor.StreamChunk, bool, error) {
if ch == nil {
return nil, true, nil
@@ -483,7 +572,7 @@ func readStreamBootstrap(ctx context.Context, ch <-chan cliproxyexecutor.StreamC
}
}
func (m *Manager) wrapStreamResult(ctx context.Context, auth *Auth, provider, routeModel string, headers http.Header, buffered []cliproxyexecutor.StreamChunk, remaining <-chan cliproxyexecutor.StreamChunk) *cliproxyexecutor.StreamResult {
func (m *Manager) wrapStreamResult(ctx context.Context, auth *Auth, provider, resultModel string, headers http.Header, buffered []cliproxyexecutor.StreamChunk, remaining <-chan cliproxyexecutor.StreamChunk) *cliproxyexecutor.StreamResult {
out := make(chan cliproxyexecutor.StreamChunk)
go func() {
defer close(out)
@@ -496,7 +585,7 @@ func (m *Manager) wrapStreamResult(ctx context.Context, auth *Auth, provider, ro
if se, ok := errors.AsType[cliproxyexecutor.StatusError](chunk.Err); ok && se != nil {
rerr.HTTPStatus = se.StatusCode()
}
m.MarkResult(ctx, Result{AuthID: auth.ID, Provider: provider, Model: routeModel, Success: false, Error: rerr})
m.MarkResult(ctx, Result{AuthID: auth.ID, Provider: provider, Model: resultModel, Success: false, Error: rerr})
}
if !forward {
return false
@@ -526,19 +615,19 @@ func (m *Manager) wrapStreamResult(ctx context.Context, auth *Auth, provider, ro
}
}
if !failed {
m.MarkResult(ctx, Result{AuthID: auth.ID, Provider: provider, Model: routeModel, Success: true})
m.MarkResult(ctx, Result{AuthID: auth.ID, Provider: provider, Model: resultModel, Success: true})
}
}()
return &cliproxyexecutor.StreamResult{Headers: headers, Chunks: out}
}
func (m *Manager) executeStreamWithModelPool(ctx context.Context, executor ProviderExecutor, auth *Auth, provider string, req cliproxyexecutor.Request, opts cliproxyexecutor.Options, routeModel string) (*cliproxyexecutor.StreamResult, error) {
func (m *Manager) executeStreamWithModelPool(ctx context.Context, executor ProviderExecutor, auth *Auth, provider string, req cliproxyexecutor.Request, opts cliproxyexecutor.Options, routeModel string, execModels []string, pooled bool) (*cliproxyexecutor.StreamResult, error) {
if executor == nil {
return nil, &Error{Code: "executor_not_found", Message: "executor not registered"}
}
execModels := m.prepareExecutionModels(auth, routeModel)
var lastErr error
for idx, execModel := range execModels {
resultModel := executionResultModel(routeModel, execModel, pooled)
execReq := req
execReq.Model = execModel
streamResult, errStream := executor.ExecuteStream(ctx, auth, execReq, opts)
@@ -550,7 +639,7 @@ func (m *Manager) executeStreamWithModelPool(ctx context.Context, executor Provi
if se, ok := errors.AsType[cliproxyexecutor.StatusError](errStream); ok && se != nil {
rerr.HTTPStatus = se.StatusCode()
}
result := Result{AuthID: auth.ID, Provider: provider, Model: routeModel, Success: false, Error: rerr}
result := Result{AuthID: auth.ID, Provider: provider, Model: resultModel, Success: false, Error: rerr}
result.RetryAfter = retryAfterFromError(errStream)
m.MarkResult(ctx, result)
if isRequestInvalidError(errStream) {
@@ -571,7 +660,7 @@ func (m *Manager) executeStreamWithModelPool(ctx context.Context, executor Provi
if se, ok := errors.AsType[cliproxyexecutor.StatusError](bootstrapErr); ok && se != nil {
rerr.HTTPStatus = se.StatusCode()
}
result := Result{AuthID: auth.ID, Provider: provider, Model: routeModel, Success: false, Error: rerr}
result := Result{AuthID: auth.ID, Provider: provider, Model: resultModel, Success: false, Error: rerr}
result.RetryAfter = retryAfterFromError(bootstrapErr)
m.MarkResult(ctx, result)
discardStreamChunks(streamResult.Chunks)
@@ -582,31 +671,33 @@ func (m *Manager) executeStreamWithModelPool(ctx context.Context, executor Provi
if se, ok := errors.AsType[cliproxyexecutor.StatusError](bootstrapErr); ok && se != nil {
rerr.HTTPStatus = se.StatusCode()
}
result := Result{AuthID: auth.ID, Provider: provider, Model: routeModel, Success: false, Error: rerr}
result := Result{AuthID: auth.ID, Provider: provider, Model: resultModel, Success: false, Error: rerr}
result.RetryAfter = retryAfterFromError(bootstrapErr)
m.MarkResult(ctx, result)
discardStreamChunks(streamResult.Chunks)
lastErr = bootstrapErr
continue
}
errCh := make(chan cliproxyexecutor.StreamChunk, 1)
errCh <- cliproxyexecutor.StreamChunk{Err: bootstrapErr}
close(errCh)
return m.wrapStreamResult(ctx, auth.Clone(), provider, routeModel, streamResult.Headers, nil, errCh), nil
rerr := &Error{Message: bootstrapErr.Error()}
if se, ok := errors.AsType[cliproxyexecutor.StatusError](bootstrapErr); ok && se != nil {
rerr.HTTPStatus = se.StatusCode()
}
result := Result{AuthID: auth.ID, Provider: provider, Model: resultModel, Success: false, Error: rerr}
result.RetryAfter = retryAfterFromError(bootstrapErr)
m.MarkResult(ctx, result)
discardStreamChunks(streamResult.Chunks)
return nil, newStreamBootstrapError(bootstrapErr, streamResult.Headers)
}
if closed && len(buffered) == 0 {
emptyErr := &Error{Code: "empty_stream", Message: "upstream stream closed before first payload", Retryable: true}
result := Result{AuthID: auth.ID, Provider: provider, Model: routeModel, Success: false, Error: emptyErr}
result := Result{AuthID: auth.ID, Provider: provider, Model: resultModel, Success: false, Error: emptyErr}
m.MarkResult(ctx, result)
if idx < len(execModels)-1 {
lastErr = emptyErr
continue
}
errCh := make(chan cliproxyexecutor.StreamChunk, 1)
errCh <- cliproxyexecutor.StreamChunk{Err: emptyErr}
close(errCh)
return m.wrapStreamResult(ctx, auth.Clone(), provider, routeModel, streamResult.Headers, nil, errCh), nil
return nil, newStreamBootstrapError(emptyErr, streamResult.Headers)
}
remaining := streamResult.Chunks
@@ -615,7 +706,7 @@ func (m *Manager) executeStreamWithModelPool(ctx context.Context, executor Provi
close(closedCh)
remaining = closedCh
}
return m.wrapStreamResult(ctx, auth.Clone(), provider, routeModel, streamResult.Headers, buffered, remaining), nil
return m.wrapStreamResult(ctx, auth.Clone(), provider, resultModel, streamResult.Headers, buffered, remaining), nil
}
if lastErr == nil {
lastErr = &Error{Code: "auth_not_found", Message: "no upstream model available"}
@@ -832,8 +923,10 @@ func (m *Manager) Update(ctx context.Context, auth *Auth) (*Auth, error) {
auth.Index = existing.Index
auth.indexAssigned = existing.indexAssigned
}
if len(auth.ModelStates) == 0 && len(existing.ModelStates) > 0 {
auth.ModelStates = existing.ModelStates
if !existing.Disabled && existing.Status != StatusDisabled && !auth.Disabled && auth.Status != StatusDisabled {
if len(auth.ModelStates) == 0 && len(existing.ModelStates) > 0 {
auth.ModelStates = existing.ModelStates
}
}
}
auth.EnsureIndex()
@@ -979,9 +1072,10 @@ func (m *Manager) executeMixedOnce(ctx context.Context, providers []string, req
routeModel := req.Model
opts = ensureRequestedModelMetadata(opts, routeModel)
tried := make(map[string]struct{})
attempted := make(map[string]struct{})
var lastErr error
for {
if maxRetryCredentials > 0 && len(tried) >= maxRetryCredentials {
if maxRetryCredentials > 0 && len(attempted) >= maxRetryCredentials {
if lastErr != nil {
return cliproxyexecutor.Response{}, lastErr
}
@@ -1006,13 +1100,18 @@ func (m *Manager) executeMixedOnce(ctx context.Context, providers []string, req
execCtx = context.WithValue(execCtx, "cliproxy.roundtripper", rt)
}
models := m.prepareExecutionModels(auth, routeModel)
models, pooled := m.preparedExecutionModels(auth, routeModel)
if len(models) == 0 {
continue
}
attempted[auth.ID] = struct{}{}
var authErr error
for _, upstreamModel := range models {
resultModel := executionResultModel(routeModel, upstreamModel, pooled)
execReq := req
execReq.Model = upstreamModel
resp, errExec := executor.Execute(execCtx, auth, execReq, opts)
result := Result{AuthID: auth.ID, Provider: provider, Model: routeModel, Success: errExec == nil}
result := Result{AuthID: auth.ID, Provider: provider, Model: resultModel, Success: errExec == nil}
if errExec != nil {
if errCtx := execCtx.Err(); errCtx != nil {
return cliproxyexecutor.Response{}, errCtx
@@ -1051,9 +1150,10 @@ func (m *Manager) executeCountMixedOnce(ctx context.Context, providers []string,
routeModel := req.Model
opts = ensureRequestedModelMetadata(opts, routeModel)
tried := make(map[string]struct{})
attempted := make(map[string]struct{})
var lastErr error
for {
if maxRetryCredentials > 0 && len(tried) >= maxRetryCredentials {
if maxRetryCredentials > 0 && len(attempted) >= maxRetryCredentials {
if lastErr != nil {
return cliproxyexecutor.Response{}, lastErr
}
@@ -1078,13 +1178,18 @@ func (m *Manager) executeCountMixedOnce(ctx context.Context, providers []string,
execCtx = context.WithValue(execCtx, "cliproxy.roundtripper", rt)
}
models := m.prepareExecutionModels(auth, routeModel)
models, pooled := m.preparedExecutionModels(auth, routeModel)
if len(models) == 0 {
continue
}
attempted[auth.ID] = struct{}{}
var authErr error
for _, upstreamModel := range models {
resultModel := executionResultModel(routeModel, upstreamModel, pooled)
execReq := req
execReq.Model = upstreamModel
resp, errExec := executor.CountTokens(execCtx, auth, execReq, opts)
result := Result{AuthID: auth.ID, Provider: provider, Model: routeModel, Success: errExec == nil}
result := Result{AuthID: auth.ID, Provider: provider, Model: resultModel, Success: errExec == nil}
if errExec != nil {
if errCtx := execCtx.Err(); errCtx != nil {
return cliproxyexecutor.Response{}, errCtx
@@ -1096,14 +1201,14 @@ func (m *Manager) executeCountMixedOnce(ctx context.Context, providers []string,
if ra := retryAfterFromError(errExec); ra != nil {
result.RetryAfter = ra
}
m.hook.OnResult(execCtx, result)
m.MarkResult(execCtx, result)
if isRequestInvalidError(errExec) {
return cliproxyexecutor.Response{}, errExec
}
authErr = errExec
continue
}
m.hook.OnResult(execCtx, result)
m.MarkResult(execCtx, result)
return resp, nil
}
if authErr != nil {
@@ -1123,10 +1228,15 @@ func (m *Manager) executeStreamMixedOnce(ctx context.Context, providers []string
routeModel := req.Model
opts = ensureRequestedModelMetadata(opts, routeModel)
tried := make(map[string]struct{})
attempted := make(map[string]struct{})
var lastErr error
for {
if maxRetryCredentials > 0 && len(tried) >= maxRetryCredentials {
if maxRetryCredentials > 0 && len(attempted) >= maxRetryCredentials {
if lastErr != nil {
var bootstrapErr *streamBootstrapError
if errors.As(lastErr, &bootstrapErr) && bootstrapErr != nil {
return streamErrorResult(bootstrapErr.Headers(), bootstrapErr.cause), nil
}
return nil, lastErr
}
return nil, &Error{Code: "auth_not_found", Message: "no auth available"}
@@ -1134,6 +1244,10 @@ func (m *Manager) executeStreamMixedOnce(ctx context.Context, providers []string
auth, executor, provider, errPick := m.pickNextMixed(ctx, providers, routeModel, opts, tried)
if errPick != nil {
if lastErr != nil {
var bootstrapErr *streamBootstrapError
if errors.As(lastErr, &bootstrapErr) && bootstrapErr != nil {
return streamErrorResult(bootstrapErr.Headers(), bootstrapErr.cause), nil
}
return nil, lastErr
}
return nil, errPick
@@ -1149,7 +1263,12 @@ func (m *Manager) executeStreamMixedOnce(ctx context.Context, providers []string
execCtx = context.WithValue(execCtx, roundTripperContextKey{}, rt)
execCtx = context.WithValue(execCtx, "cliproxy.roundtripper", rt)
}
streamResult, errStream := m.executeStreamWithModelPool(execCtx, executor, auth, provider, req, opts, routeModel)
models, pooled := m.preparedExecutionModels(auth, routeModel)
if len(models) == 0 {
continue
}
attempted[auth.ID] = struct{}{}
streamResult, errStream := m.executeStreamWithModelPool(execCtx, executor, auth, provider, req, opts, routeModel, models, pooled)
if errStream != nil {
if errCtx := execCtx.Err(); errCtx != nil {
return nil, errCtx
@@ -1615,70 +1734,79 @@ func (m *Manager) MarkResult(ctx context.Context, result Result) {
}
} else {
if result.Model != "" {
state := ensureModelState(auth, result.Model)
state.Unavailable = true
state.Status = StatusError
state.UpdatedAt = now
if result.Error != nil {
state.LastError = cloneError(result.Error)
state.StatusMessage = result.Error.Message
auth.LastError = cloneError(result.Error)
auth.StatusMessage = result.Error.Message
}
if !isRequestScopedNotFoundResultError(result.Error) {
state := ensureModelState(auth, result.Model)
state.Unavailable = true
state.Status = StatusError
state.UpdatedAt = now
if result.Error != nil {
state.LastError = cloneError(result.Error)
state.StatusMessage = result.Error.Message
auth.LastError = cloneError(result.Error)
auth.StatusMessage = result.Error.Message
}
statusCode := statusCodeFromResult(result.Error)
switch statusCode {
case 401:
next := now.Add(30 * time.Minute)
state.NextRetryAfter = next
suspendReason = "unauthorized"
shouldSuspendModel = true
case 402, 403:
next := now.Add(30 * time.Minute)
state.NextRetryAfter = next
suspendReason = "payment_required"
shouldSuspendModel = true
case 404:
next := now.Add(12 * time.Hour)
state.NextRetryAfter = next
suspendReason = "not_found"
shouldSuspendModel = true
case 429:
var next time.Time
backoffLevel := state.Quota.BackoffLevel
if result.RetryAfter != nil {
next = now.Add(*result.RetryAfter)
} else {
cooldown, nextLevel := nextQuotaCooldown(backoffLevel, quotaCooldownDisabledForAuth(auth))
if cooldown > 0 {
next = now.Add(cooldown)
}
backoffLevel = nextLevel
}
state.NextRetryAfter = next
state.Quota = QuotaState{
Exceeded: true,
Reason: "quota",
NextRecoverAt: next,
BackoffLevel: backoffLevel,
}
suspendReason = "quota"
shouldSuspendModel = true
setModelQuota = true
case 408, 500, 502, 503, 504:
if quotaCooldownDisabledForAuth(auth) {
state.NextRetryAfter = time.Time{}
} else {
next := now.Add(1 * time.Minute)
statusCode := statusCodeFromResult(result.Error)
if isModelSupportResultError(result.Error) {
next := now.Add(12 * time.Hour)
state.NextRetryAfter = next
suspendReason = "model_not_supported"
shouldSuspendModel = true
} else {
switch statusCode {
case 401:
next := now.Add(30 * time.Minute)
state.NextRetryAfter = next
suspendReason = "unauthorized"
shouldSuspendModel = true
case 402, 403:
next := now.Add(30 * time.Minute)
state.NextRetryAfter = next
suspendReason = "payment_required"
shouldSuspendModel = true
case 404:
next := now.Add(12 * time.Hour)
state.NextRetryAfter = next
suspendReason = "not_found"
shouldSuspendModel = true
case 429:
var next time.Time
backoffLevel := state.Quota.BackoffLevel
if result.RetryAfter != nil {
next = now.Add(*result.RetryAfter)
} else {
cooldown, nextLevel := nextQuotaCooldown(backoffLevel, quotaCooldownDisabledForAuth(auth))
if cooldown > 0 {
next = now.Add(cooldown)
}
backoffLevel = nextLevel
}
state.NextRetryAfter = next
state.Quota = QuotaState{
Exceeded: true,
Reason: "quota",
NextRecoverAt: next,
BackoffLevel: backoffLevel,
}
suspendReason = "quota"
shouldSuspendModel = true
setModelQuota = true
case 408, 500, 502, 503, 504:
if quotaCooldownDisabledForAuth(auth) {
state.NextRetryAfter = time.Time{}
} else {
next := now.Add(1 * time.Minute)
state.NextRetryAfter = next
}
default:
state.NextRetryAfter = time.Time{}
}
}
default:
state.NextRetryAfter = time.Time{}
}
auth.Status = StatusError
auth.UpdatedAt = now
updateAggregatedAvailability(auth, now)
auth.Status = StatusError
auth.UpdatedAt = now
updateAggregatedAvailability(auth, now)
}
} else {
applyAuthFailureState(auth, result.Error, result.RetryAfter, now)
}
@@ -1883,18 +2011,89 @@ func statusCodeFromResult(err *Error) int {
return err.StatusCode()
}
func isModelSupportErrorMessage(message string) bool {
lower := strings.ToLower(strings.TrimSpace(message))
if lower == "" {
return false
}
patterns := [...]string{
"model_not_supported",
"requested model is not supported",
"requested model is unsupported",
"requested model is unavailable",
"model is not supported",
"model not supported",
"unsupported model",
"model unavailable",
"not available for your plan",
"not available for your account",
}
for _, pattern := range patterns {
if strings.Contains(lower, pattern) {
return true
}
}
return false
}
func isModelSupportError(err error) bool {
if err == nil {
return false
}
status := statusCodeFromError(err)
if status != http.StatusBadRequest && status != http.StatusUnprocessableEntity {
return false
}
return isModelSupportErrorMessage(err.Error())
}
func isModelSupportResultError(err *Error) bool {
if err == nil {
return false
}
status := statusCodeFromResult(err)
if status != http.StatusBadRequest && status != http.StatusUnprocessableEntity {
return false
}
return isModelSupportErrorMessage(err.Message)
}
func isRequestScopedNotFoundMessage(message string) bool {
if message == "" {
return false
}
lower := strings.ToLower(message)
return strings.Contains(lower, "item with id") &&
strings.Contains(lower, "not found") &&
strings.Contains(lower, "items are not persisted when `store` is set to false")
}
func isRequestScopedNotFoundResultError(err *Error) bool {
if err == nil || statusCodeFromResult(err) != http.StatusNotFound {
return false
}
return isRequestScopedNotFoundMessage(err.Message)
}
// isRequestInvalidError returns true if the error represents a client request
// error that should not be retried. Specifically, it treats 400 responses with
// "invalid_request_error" and all 422 responses as request-shape failures,
// where switching auths or pooled upstream models will not help.
// "invalid_request_error", request-scoped 404 item misses caused by `store=false`,
// and all 422 responses as request-shape failures, where switching auths or
// pooled upstream models will not help. Model-support errors are excluded so
// routing can fall through to another auth or upstream.
func isRequestInvalidError(err error) bool {
if err == nil {
return false
}
if isModelSupportError(err) {
return false
}
status := statusCodeFromError(err)
switch status {
case http.StatusBadRequest:
return strings.Contains(err.Error(), "invalid_request_error")
case http.StatusNotFound:
return isRequestScopedNotFoundMessage(err.Error())
case http.StatusUnprocessableEntity:
return true
default:
@@ -1906,6 +2105,9 @@ func applyAuthFailureState(auth *Auth, resultErr *Error, retryAfter *time.Durati
if auth == nil {
return
}
if isRequestScopedNotFoundResultError(resultErr) {
return
}
auth.Unavailable = true
auth.Status = StatusError
auth.UpdatedAt = now
@@ -12,6 +12,8 @@ import (
cliproxyexecutor "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/executor"
)
const requestScopedNotFoundMessage = "Item with id 'rs_0b5f3eb6f51f175c0169ca74e4a85881998539920821603a74' not found. Items are not persisted when `store` is set to false. Try again with `store` set to true, or remove this item from your input."
func TestManager_ShouldRetryAfterError_RespectsAuthRequestRetryOverride(t *testing.T) {
m := NewManager(nil, nil, nil)
m.SetRetryConfig(3, 30*time.Second, 0)
@@ -108,6 +110,76 @@ func (e *credentialRetryLimitExecutor) Calls() int {
return e.calls
}
type authFallbackExecutor struct {
id string
mu sync.Mutex
executeCalls []string
streamCalls []string
executeErrors map[string]error
streamFirstErrors map[string]error
}
func (e *authFallbackExecutor) Identifier() string {
return e.id
}
func (e *authFallbackExecutor) Execute(_ context.Context, auth *Auth, _ cliproxyexecutor.Request, _ cliproxyexecutor.Options) (cliproxyexecutor.Response, error) {
e.mu.Lock()
e.executeCalls = append(e.executeCalls, auth.ID)
err := e.executeErrors[auth.ID]
e.mu.Unlock()
if err != nil {
return cliproxyexecutor.Response{}, err
}
return cliproxyexecutor.Response{Payload: []byte(auth.ID)}, nil
}
func (e *authFallbackExecutor) ExecuteStream(_ context.Context, auth *Auth, _ cliproxyexecutor.Request, _ cliproxyexecutor.Options) (*cliproxyexecutor.StreamResult, error) {
e.mu.Lock()
e.streamCalls = append(e.streamCalls, auth.ID)
err := e.streamFirstErrors[auth.ID]
e.mu.Unlock()
ch := make(chan cliproxyexecutor.StreamChunk, 1)
if err != nil {
ch <- cliproxyexecutor.StreamChunk{Err: err}
close(ch)
return &cliproxyexecutor.StreamResult{Headers: http.Header{"X-Auth": {auth.ID}}, Chunks: ch}, nil
}
ch <- cliproxyexecutor.StreamChunk{Payload: []byte(auth.ID)}
close(ch)
return &cliproxyexecutor.StreamResult{Headers: http.Header{"X-Auth": {auth.ID}}, Chunks: ch}, nil
}
func (e *authFallbackExecutor) Refresh(_ context.Context, auth *Auth) (*Auth, error) {
return auth, nil
}
func (e *authFallbackExecutor) CountTokens(context.Context, *Auth, cliproxyexecutor.Request, cliproxyexecutor.Options) (cliproxyexecutor.Response, error) {
return cliproxyexecutor.Response{}, &Error{HTTPStatus: 500, Message: "not implemented"}
}
func (e *authFallbackExecutor) HttpRequest(context.Context, *Auth, *http.Request) (*http.Response, error) {
return nil, nil
}
func (e *authFallbackExecutor) ExecuteCalls() []string {
e.mu.Lock()
defer e.mu.Unlock()
out := make([]string, len(e.executeCalls))
copy(out, e.executeCalls)
return out
}
func (e *authFallbackExecutor) StreamCalls() []string {
e.mu.Lock()
defer e.mu.Unlock()
out := make([]string, len(e.streamCalls))
copy(out, e.streamCalls)
return out
}
func newCredentialRetryLimitTestManager(t *testing.T, maxRetryCredentials int) (*Manager, *credentialRetryLimitExecutor) {
t.Helper()
@@ -191,6 +263,153 @@ func TestManager_MaxRetryCredentials_LimitsCrossCredentialRetries(t *testing.T)
}
}
func TestManager_ModelSupportBadRequest_FallsBackAndSuspendsAuth(t *testing.T) {
m := NewManager(nil, nil, nil)
executor := &authFallbackExecutor{
id: "claude",
executeErrors: map[string]error{
"aa-bad-auth": &Error{
HTTPStatus: http.StatusBadRequest,
Message: "invalid_request_error: The requested model is not supported.",
},
},
}
m.RegisterExecutor(executor)
model := "claude-opus-4-6"
badAuth := &Auth{ID: "aa-bad-auth", Provider: "claude"}
goodAuth := &Auth{ID: "bb-good-auth", Provider: "claude"}
reg := registry.GetGlobalRegistry()
reg.RegisterClient(badAuth.ID, "claude", []*registry.ModelInfo{{ID: model}})
reg.RegisterClient(goodAuth.ID, "claude", []*registry.ModelInfo{{ID: model}})
t.Cleanup(func() {
reg.UnregisterClient(badAuth.ID)
reg.UnregisterClient(goodAuth.ID)
})
if _, errRegister := m.Register(context.Background(), badAuth); errRegister != nil {
t.Fatalf("register bad auth: %v", errRegister)
}
if _, errRegister := m.Register(context.Background(), goodAuth); errRegister != nil {
t.Fatalf("register good auth: %v", errRegister)
}
request := cliproxyexecutor.Request{Model: model}
for i := 0; i < 2; i++ {
resp, errExecute := m.Execute(context.Background(), []string{"claude"}, request, cliproxyexecutor.Options{})
if errExecute != nil {
t.Fatalf("execute %d error = %v, want success", i, errExecute)
}
if string(resp.Payload) != goodAuth.ID {
t.Fatalf("execute %d payload = %q, want %q", i, string(resp.Payload), goodAuth.ID)
}
}
got := executor.ExecuteCalls()
want := []string{badAuth.ID, goodAuth.ID, goodAuth.ID}
if len(got) != len(want) {
t.Fatalf("execute calls = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("execute call %d auth = %q, want %q", i, got[i], want[i])
}
}
updatedBad, ok := m.GetByID(badAuth.ID)
if !ok || updatedBad == nil {
t.Fatalf("expected bad auth to remain registered")
}
state := updatedBad.ModelStates[model]
if state == nil {
t.Fatalf("expected model state for %q", model)
}
if !state.Unavailable {
t.Fatalf("expected bad auth model state to be unavailable")
}
if state.NextRetryAfter.IsZero() {
t.Fatalf("expected bad auth model state cooldown to be set")
}
}
func TestManagerExecuteStream_ModelSupportBadRequestFallsBackAndSuspendsAuth(t *testing.T) {
m := NewManager(nil, nil, nil)
executor := &authFallbackExecutor{
id: "claude",
streamFirstErrors: map[string]error{
"aa-bad-auth": &Error{
HTTPStatus: http.StatusBadRequest,
Message: "invalid_request_error: The requested model is not supported.",
},
},
}
m.RegisterExecutor(executor)
model := "claude-opus-4-6"
badAuth := &Auth{ID: "aa-bad-auth", Provider: "claude"}
goodAuth := &Auth{ID: "bb-good-auth", Provider: "claude"}
reg := registry.GetGlobalRegistry()
reg.RegisterClient(badAuth.ID, "claude", []*registry.ModelInfo{{ID: model}})
reg.RegisterClient(goodAuth.ID, "claude", []*registry.ModelInfo{{ID: model}})
t.Cleanup(func() {
reg.UnregisterClient(badAuth.ID)
reg.UnregisterClient(goodAuth.ID)
})
if _, errRegister := m.Register(context.Background(), badAuth); errRegister != nil {
t.Fatalf("register bad auth: %v", errRegister)
}
if _, errRegister := m.Register(context.Background(), goodAuth); errRegister != nil {
t.Fatalf("register good auth: %v", errRegister)
}
request := cliproxyexecutor.Request{Model: model}
for i := 0; i < 2; i++ {
streamResult, errExecute := m.ExecuteStream(context.Background(), []string{"claude"}, request, cliproxyexecutor.Options{})
if errExecute != nil {
t.Fatalf("execute stream %d error = %v, want success", i, errExecute)
}
var payload []byte
for chunk := range streamResult.Chunks {
if chunk.Err != nil {
t.Fatalf("execute stream %d chunk error = %v, want success", i, chunk.Err)
}
payload = append(payload, chunk.Payload...)
}
if string(payload) != goodAuth.ID {
t.Fatalf("execute stream %d payload = %q, want %q", i, string(payload), goodAuth.ID)
}
}
got := executor.StreamCalls()
want := []string{badAuth.ID, goodAuth.ID, goodAuth.ID}
if len(got) != len(want) {
t.Fatalf("stream calls = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("stream call %d auth = %q, want %q", i, got[i], want[i])
}
}
updatedBad, ok := m.GetByID(badAuth.ID)
if !ok || updatedBad == nil {
t.Fatalf("expected bad auth to remain registered")
}
state := updatedBad.ModelStates[model]
if state == nil {
t.Fatalf("expected model state for %q", model)
}
if !state.Unavailable {
t.Fatalf("expected bad auth model state to be unavailable")
}
if state.NextRetryAfter.IsZero() {
t.Fatalf("expected bad auth model state cooldown to be set")
}
}
func TestManager_MarkResult_RespectsAuthDisableCoolingOverride(t *testing.T) {
prev := quotaCooldownDisabled.Load()
quotaCooldownDisabled.Store(false)
@@ -230,3 +449,114 @@ func TestManager_MarkResult_RespectsAuthDisableCoolingOverride(t *testing.T) {
t.Fatalf("expected NextRetryAfter to be zero when disable_cooling=true, got %v", state.NextRetryAfter)
}
}
func TestManager_MarkResult_RequestScopedNotFoundDoesNotCooldownAuth(t *testing.T) {
m := NewManager(nil, nil, nil)
auth := &Auth{
ID: "auth-1",
Provider: "openai",
}
if _, errRegister := m.Register(context.Background(), auth); errRegister != nil {
t.Fatalf("register auth: %v", errRegister)
}
model := "gpt-4.1"
m.MarkResult(context.Background(), Result{
AuthID: auth.ID,
Provider: auth.Provider,
Model: model,
Success: false,
Error: &Error{
HTTPStatus: http.StatusNotFound,
Message: requestScopedNotFoundMessage,
},
})
updated, ok := m.GetByID(auth.ID)
if !ok || updated == nil {
t.Fatalf("expected auth to be present")
}
if updated.Unavailable {
t.Fatalf("expected request-scoped 404 to keep auth available")
}
if !updated.NextRetryAfter.IsZero() {
t.Fatalf("expected request-scoped 404 to keep auth cooldown unset, got %v", updated.NextRetryAfter)
}
if state := updated.ModelStates[model]; state != nil {
t.Fatalf("expected request-scoped 404 to avoid model cooldown state, got %#v", state)
}
}
func TestManager_RequestScopedNotFoundStopsRetryWithoutSuspendingAuth(t *testing.T) {
m := NewManager(nil, nil, nil)
executor := &authFallbackExecutor{
id: "openai",
executeErrors: map[string]error{
"aa-bad-auth": &Error{
HTTPStatus: http.StatusNotFound,
Message: requestScopedNotFoundMessage,
},
},
}
m.RegisterExecutor(executor)
model := "gpt-4.1"
badAuth := &Auth{ID: "aa-bad-auth", Provider: "openai"}
goodAuth := &Auth{ID: "bb-good-auth", Provider: "openai"}
reg := registry.GetGlobalRegistry()
reg.RegisterClient(badAuth.ID, "openai", []*registry.ModelInfo{{ID: model}})
reg.RegisterClient(goodAuth.ID, "openai", []*registry.ModelInfo{{ID: model}})
t.Cleanup(func() {
reg.UnregisterClient(badAuth.ID)
reg.UnregisterClient(goodAuth.ID)
})
if _, errRegister := m.Register(context.Background(), badAuth); errRegister != nil {
t.Fatalf("register bad auth: %v", errRegister)
}
if _, errRegister := m.Register(context.Background(), goodAuth); errRegister != nil {
t.Fatalf("register good auth: %v", errRegister)
}
_, errExecute := m.Execute(context.Background(), []string{"openai"}, cliproxyexecutor.Request{Model: model}, cliproxyexecutor.Options{})
if errExecute == nil {
t.Fatal("expected request-scoped not-found error")
}
errResult, ok := errExecute.(*Error)
if !ok {
t.Fatalf("expected *Error, got %T", errExecute)
}
if errResult.HTTPStatus != http.StatusNotFound {
t.Fatalf("status = %d, want %d", errResult.HTTPStatus, http.StatusNotFound)
}
if errResult.Message != requestScopedNotFoundMessage {
t.Fatalf("message = %q, want %q", errResult.Message, requestScopedNotFoundMessage)
}
got := executor.ExecuteCalls()
want := []string{badAuth.ID}
if len(got) != len(want) {
t.Fatalf("execute calls = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("execute call %d auth = %q, want %q", i, got[i], want[i])
}
}
updatedBad, ok := m.GetByID(badAuth.ID)
if !ok || updatedBad == nil {
t.Fatalf("expected bad auth to remain registered")
}
if updatedBad.Unavailable {
t.Fatalf("expected request-scoped 404 to keep bad auth available")
}
if !updatedBad.NextRetryAfter.IsZero() {
t.Fatalf("expected request-scoped 404 to keep bad auth cooldown unset, got %v", updatedBad.NextRetryAfter)
}
if state := updatedBad.ModelStates[model]; state != nil {
t.Fatalf("expected request-scoped 404 to avoid bad auth model cooldown state, got %#v", state)
}
}
+155
View File
@@ -47,3 +47,158 @@ func TestManager_Update_PreservesModelStates(t *testing.T) {
t.Fatalf("expected BackoffLevel to be %d, got %d", backoffLevel, state.Quota.BackoffLevel)
}
}
func TestManager_Update_DisabledExistingDoesNotInheritModelStates(t *testing.T) {
m := NewManager(nil, nil, nil)
// Register a disabled auth with existing ModelStates.
if _, err := m.Register(context.Background(), &Auth{
ID: "auth-disabled",
Provider: "claude",
Disabled: true,
Status: StatusDisabled,
ModelStates: map[string]*ModelState{
"stale-model": {
Quota: QuotaState{BackoffLevel: 5},
},
},
}); err != nil {
t.Fatalf("register auth: %v", err)
}
// Update with empty ModelStates — should NOT inherit stale states.
if _, err := m.Update(context.Background(), &Auth{
ID: "auth-disabled",
Provider: "claude",
Disabled: true,
Status: StatusDisabled,
}); err != nil {
t.Fatalf("update auth: %v", err)
}
updated, ok := m.GetByID("auth-disabled")
if !ok || updated == nil {
t.Fatalf("expected auth to be present")
}
if len(updated.ModelStates) != 0 {
t.Fatalf("expected disabled auth NOT to inherit ModelStates, got %d entries", len(updated.ModelStates))
}
}
func TestManager_Update_ActiveToDisabledDoesNotInheritModelStates(t *testing.T) {
m := NewManager(nil, nil, nil)
// Register an active auth with ModelStates (simulates existing live auth).
if _, err := m.Register(context.Background(), &Auth{
ID: "auth-a2d",
Provider: "claude",
Status: StatusActive,
ModelStates: map[string]*ModelState{
"stale-model": {
Quota: QuotaState{BackoffLevel: 9},
},
},
}); err != nil {
t.Fatalf("register auth: %v", err)
}
// File watcher deletes config → synthesizes Disabled=true auth → Update.
// Even though existing is active, incoming auth is disabled → skip inheritance.
if _, err := m.Update(context.Background(), &Auth{
ID: "auth-a2d",
Provider: "claude",
Disabled: true,
Status: StatusDisabled,
}); err != nil {
t.Fatalf("update auth: %v", err)
}
updated, ok := m.GetByID("auth-a2d")
if !ok || updated == nil {
t.Fatalf("expected auth to be present")
}
if len(updated.ModelStates) != 0 {
t.Fatalf("expected active→disabled transition NOT to inherit ModelStates, got %d entries", len(updated.ModelStates))
}
}
func TestManager_Update_DisabledToActiveDoesNotInheritStaleModelStates(t *testing.T) {
m := NewManager(nil, nil, nil)
// Register a disabled auth with stale ModelStates.
if _, err := m.Register(context.Background(), &Auth{
ID: "auth-d2a",
Provider: "claude",
Disabled: true,
Status: StatusDisabled,
ModelStates: map[string]*ModelState{
"stale-model": {
Quota: QuotaState{BackoffLevel: 4},
},
},
}); err != nil {
t.Fatalf("register auth: %v", err)
}
// Re-enable: incoming auth is active, existing is disabled → skip inheritance.
if _, err := m.Update(context.Background(), &Auth{
ID: "auth-d2a",
Provider: "claude",
Status: StatusActive,
}); err != nil {
t.Fatalf("update auth: %v", err)
}
updated, ok := m.GetByID("auth-d2a")
if !ok || updated == nil {
t.Fatalf("expected auth to be present")
}
if len(updated.ModelStates) != 0 {
t.Fatalf("expected disabled→active transition NOT to inherit stale ModelStates, got %d entries", len(updated.ModelStates))
}
}
func TestManager_Update_ActiveInheritsModelStates(t *testing.T) {
m := NewManager(nil, nil, nil)
model := "active-model"
backoffLevel := 3
// Register an active auth with ModelStates.
if _, err := m.Register(context.Background(), &Auth{
ID: "auth-active",
Provider: "claude",
Status: StatusActive,
ModelStates: map[string]*ModelState{
model: {
Quota: QuotaState{BackoffLevel: backoffLevel},
},
},
}); err != nil {
t.Fatalf("register auth: %v", err)
}
// Update with empty ModelStates — both sides active → SHOULD inherit.
if _, err := m.Update(context.Background(), &Auth{
ID: "auth-active",
Provider: "claude",
Status: StatusActive,
}); err != nil {
t.Fatalf("update auth: %v", err)
}
updated, ok := m.GetByID("auth-active")
if !ok || updated == nil {
t.Fatalf("expected auth to be present")
}
if len(updated.ModelStates) == 0 {
t.Fatalf("expected active auth to inherit ModelStates")
}
state := updated.ModelStates[model]
if state == nil {
t.Fatalf("expected model state to be present")
}
if state.Quota.BackoffLevel != backoffLevel {
t.Fatalf("expected BackoffLevel to be %d, got %d", backoffLevel, state.Quota.BackoffLevel)
}
}
@@ -3,6 +3,7 @@ package auth
import (
"context"
"net/http"
"strings"
"sync"
"testing"
@@ -116,6 +117,47 @@ func (e *openAICompatPoolExecutor) StreamModels() []string {
return out
}
type authScopedOpenAICompatPoolExecutor struct {
id string
mu sync.Mutex
executeCalls []string
}
func (e *authScopedOpenAICompatPoolExecutor) Identifier() string { return e.id }
func (e *authScopedOpenAICompatPoolExecutor) Execute(_ context.Context, auth *Auth, req cliproxyexecutor.Request, _ cliproxyexecutor.Options) (cliproxyexecutor.Response, error) {
call := auth.ID + "|" + req.Model
e.mu.Lock()
e.executeCalls = append(e.executeCalls, call)
e.mu.Unlock()
return cliproxyexecutor.Response{Payload: []byte(call)}, nil
}
func (e *authScopedOpenAICompatPoolExecutor) ExecuteStream(context.Context, *Auth, cliproxyexecutor.Request, cliproxyexecutor.Options) (*cliproxyexecutor.StreamResult, error) {
return nil, &Error{HTTPStatus: http.StatusNotImplemented, Message: "ExecuteStream not implemented"}
}
func (e *authScopedOpenAICompatPoolExecutor) Refresh(_ context.Context, auth *Auth) (*Auth, error) {
return auth, nil
}
func (e *authScopedOpenAICompatPoolExecutor) CountTokens(context.Context, *Auth, cliproxyexecutor.Request, cliproxyexecutor.Options) (cliproxyexecutor.Response, error) {
return cliproxyexecutor.Response{}, &Error{HTTPStatus: http.StatusNotImplemented, Message: "CountTokens not implemented"}
}
func (e *authScopedOpenAICompatPoolExecutor) HttpRequest(context.Context, *Auth, *http.Request) (*http.Response, error) {
return nil, &Error{HTTPStatus: http.StatusNotImplemented, Message: "HttpRequest not implemented"}
}
func (e *authScopedOpenAICompatPoolExecutor) ExecuteCalls() []string {
e.mu.Lock()
defer e.mu.Unlock()
out := make([]string, len(e.executeCalls))
copy(out, e.executeCalls)
return out
}
func newOpenAICompatPoolTestManager(t *testing.T, alias string, models []internalconfig.OpenAICompatibilityModel, executor *openAICompatPoolExecutor) *Manager {
t.Helper()
cfg := &internalconfig.Config{
@@ -153,6 +195,21 @@ func newOpenAICompatPoolTestManager(t *testing.T, alias string, models []interna
return m
}
func readOpenAICompatStreamPayload(t *testing.T, streamResult *cliproxyexecutor.StreamResult) string {
t.Helper()
if streamResult == nil {
t.Fatal("expected stream result")
}
var payload []byte
for chunk := range streamResult.Chunks {
if chunk.Err != nil {
t.Fatalf("unexpected stream error: %v", chunk.Err)
}
payload = append(payload, chunk.Payload...)
}
return string(payload)
}
func TestManagerExecuteCount_OpenAICompatAliasPoolStopsOnInvalidRequest(t *testing.T) {
alias := "claude-opus-4.66"
invalidErr := &Error{HTTPStatus: http.StatusUnprocessableEntity, Message: "unprocessable entity"}
@@ -243,6 +300,87 @@ func TestManagerExecute_OpenAICompatAliasPoolStopsOnBadRequest(t *testing.T) {
t.Fatalf("execute calls = %v, want only first invalid model", got)
}
}
func TestManagerExecute_OpenAICompatAliasPoolFallsBackOnModelSupportBadRequest(t *testing.T) {
alias := "claude-opus-4.66"
modelSupportErr := &Error{
HTTPStatus: http.StatusBadRequest,
Message: "invalid_request_error: The requested model is not supported.",
}
executor := &openAICompatPoolExecutor{
id: "pool",
executeErrors: map[string]error{"qwen3.5-plus": modelSupportErr},
}
m := newOpenAICompatPoolTestManager(t, alias, []internalconfig.OpenAICompatibilityModel{
{Name: "qwen3.5-plus", Alias: alias},
{Name: "glm-5", Alias: alias},
}, executor)
resp, err := m.Execute(context.Background(), []string{"pool"}, cliproxyexecutor.Request{Model: alias}, cliproxyexecutor.Options{})
if err != nil {
t.Fatalf("execute error = %v, want fallback success", err)
}
if string(resp.Payload) != "glm-5" {
t.Fatalf("payload = %q, want %q", string(resp.Payload), "glm-5")
}
got := executor.ExecuteModels()
want := []string{"qwen3.5-plus", "glm-5"}
if len(got) != len(want) {
t.Fatalf("execute calls = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("execute call %d model = %q, want %q", i, got[i], want[i])
}
}
updated, ok := m.GetByID("pool-auth-" + t.Name())
if !ok || updated == nil {
t.Fatalf("expected auth to remain registered")
}
state := updated.ModelStates["qwen3.5-plus"]
if state == nil {
t.Fatalf("expected suspended upstream model state")
}
if !state.Unavailable || state.NextRetryAfter.IsZero() {
t.Fatalf("expected upstream model suspension, got %+v", state)
}
}
func TestManagerExecute_OpenAICompatAliasPoolFallsBackOnModelSupportUnprocessableEntity(t *testing.T) {
alias := "claude-opus-4.66"
modelSupportErr := &Error{
HTTPStatus: http.StatusUnprocessableEntity,
Message: "The requested model is not supported.",
}
executor := &openAICompatPoolExecutor{
id: "pool",
executeErrors: map[string]error{"qwen3.5-plus": modelSupportErr},
}
m := newOpenAICompatPoolTestManager(t, alias, []internalconfig.OpenAICompatibilityModel{
{Name: "qwen3.5-plus", Alias: alias},
{Name: "glm-5", Alias: alias},
}, executor)
resp, err := m.Execute(context.Background(), []string{"pool"}, cliproxyexecutor.Request{Model: alias}, cliproxyexecutor.Options{})
if err != nil {
t.Fatalf("execute error = %v, want fallback success", err)
}
if string(resp.Payload) != "glm-5" {
t.Fatalf("payload = %q, want %q", string(resp.Payload), "glm-5")
}
got := executor.ExecuteModels()
want := []string{"qwen3.5-plus", "glm-5"}
if len(got) != len(want) {
t.Fatalf("execute calls = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("execute call %d model = %q, want %q", i, got[i], want[i])
}
}
}
func TestManagerExecute_OpenAICompatAliasPoolFallsBackWithinSameAuth(t *testing.T) {
alias := "claude-opus-4.66"
executor := &openAICompatPoolExecutor{
@@ -364,6 +502,84 @@ func TestManagerExecuteStream_OpenAICompatAliasPoolStopsOnInvalidRequest(t *test
t.Fatalf("stream calls = %v, want only first invalid model", got)
}
}
func TestManagerExecute_OpenAICompatAliasPoolSkipsSuspendedUpstreamOnLaterRequests(t *testing.T) {
alias := "claude-opus-4.66"
modelSupportErr := &Error{
HTTPStatus: http.StatusBadRequest,
Message: "invalid_request_error: The requested model is not supported.",
}
executor := &openAICompatPoolExecutor{
id: "pool",
executeErrors: map[string]error{"qwen3.5-plus": modelSupportErr},
}
m := newOpenAICompatPoolTestManager(t, alias, []internalconfig.OpenAICompatibilityModel{
{Name: "qwen3.5-plus", Alias: alias},
{Name: "glm-5", Alias: alias},
}, executor)
for i := 0; i < 3; i++ {
resp, err := m.Execute(context.Background(), []string{"pool"}, cliproxyexecutor.Request{Model: alias}, cliproxyexecutor.Options{})
if err != nil {
t.Fatalf("execute %d: %v", i, err)
}
if string(resp.Payload) != "glm-5" {
t.Fatalf("execute %d payload = %q, want %q", i, string(resp.Payload), "glm-5")
}
}
got := executor.ExecuteModels()
want := []string{"qwen3.5-plus", "glm-5", "glm-5", "glm-5"}
if len(got) != len(want) {
t.Fatalf("execute calls = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("execute call %d model = %q, want %q", i, got[i], want[i])
}
}
}
func TestManagerExecuteStream_OpenAICompatAliasPoolSkipsSuspendedUpstreamOnLaterRequests(t *testing.T) {
alias := "claude-opus-4.66"
modelSupportErr := &Error{
HTTPStatus: http.StatusUnprocessableEntity,
Message: "The requested model is not supported.",
}
executor := &openAICompatPoolExecutor{
id: "pool",
streamFirstErrors: map[string]error{"qwen3.5-plus": modelSupportErr},
}
m := newOpenAICompatPoolTestManager(t, alias, []internalconfig.OpenAICompatibilityModel{
{Name: "qwen3.5-plus", Alias: alias},
{Name: "glm-5", Alias: alias},
}, executor)
for i := 0; i < 3; i++ {
streamResult, err := m.ExecuteStream(context.Background(), []string{"pool"}, cliproxyexecutor.Request{Model: alias}, cliproxyexecutor.Options{})
if err != nil {
t.Fatalf("execute stream %d: %v", i, err)
}
if payload := readOpenAICompatStreamPayload(t, streamResult); payload != "glm-5" {
t.Fatalf("execute stream %d payload = %q, want %q", i, payload, "glm-5")
}
if gotHeader := streamResult.Headers.Get("X-Model"); gotHeader != "glm-5" {
t.Fatalf("execute stream %d header X-Model = %q, want %q", i, gotHeader, "glm-5")
}
}
got := executor.StreamModels()
want := []string{"qwen3.5-plus", "glm-5", "glm-5", "glm-5"}
if len(got) != len(want) {
t.Fatalf("stream calls = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("stream call %d model = %q, want %q", i, got[i], want[i])
}
}
}
func TestManagerExecuteCount_OpenAICompatAliasPoolRotatesWithinAuth(t *testing.T) {
alias := "claude-opus-4.66"
executor := &openAICompatPoolExecutor{id: "pool"}
@@ -391,6 +607,127 @@ func TestManagerExecuteCount_OpenAICompatAliasPoolRotatesWithinAuth(t *testing.T
}
}
func TestManagerExecuteCount_OpenAICompatAliasPoolSkipsSuspendedUpstreamOnLaterRequests(t *testing.T) {
alias := "claude-opus-4.66"
modelSupportErr := &Error{
HTTPStatus: http.StatusBadRequest,
Message: "invalid_request_error: The requested model is unsupported.",
}
executor := &openAICompatPoolExecutor{
id: "pool",
countErrors: map[string]error{"qwen3.5-plus": modelSupportErr},
}
m := newOpenAICompatPoolTestManager(t, alias, []internalconfig.OpenAICompatibilityModel{
{Name: "qwen3.5-plus", Alias: alias},
{Name: "glm-5", Alias: alias},
}, executor)
for i := 0; i < 3; i++ {
resp, err := m.ExecuteCount(context.Background(), []string{"pool"}, cliproxyexecutor.Request{Model: alias}, cliproxyexecutor.Options{})
if err != nil {
t.Fatalf("execute count %d: %v", i, err)
}
if string(resp.Payload) != "glm-5" {
t.Fatalf("execute count %d payload = %q, want %q", i, string(resp.Payload), "glm-5")
}
}
got := executor.CountModels()
want := []string{"qwen3.5-plus", "glm-5", "glm-5", "glm-5"}
if len(got) != len(want) {
t.Fatalf("count calls = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("count call %d model = %q, want %q", i, got[i], want[i])
}
}
}
func TestManagerExecute_OpenAICompatAliasPoolBlockedAuthDoesNotConsumeRetryBudget(t *testing.T) {
alias := "claude-opus-4.66"
cfg := &internalconfig.Config{
OpenAICompatibility: []internalconfig.OpenAICompatibility{{
Name: "pool",
Models: []internalconfig.OpenAICompatibilityModel{
{Name: "qwen3.5-plus", Alias: alias},
{Name: "glm-5", Alias: alias},
},
}},
}
m := NewManager(nil, nil, nil)
m.SetConfig(cfg)
m.SetRetryConfig(0, 0, 1)
executor := &authScopedOpenAICompatPoolExecutor{id: "pool"}
m.RegisterExecutor(executor)
badAuth := &Auth{
ID: "aa-blocked-auth",
Provider: "pool",
Status: StatusActive,
Attributes: map[string]string{
"api_key": "bad-key",
"compat_name": "pool",
"provider_key": "pool",
},
}
goodAuth := &Auth{
ID: "bb-good-auth",
Provider: "pool",
Status: StatusActive,
Attributes: map[string]string{
"api_key": "good-key",
"compat_name": "pool",
"provider_key": "pool",
},
}
if _, err := m.Register(context.Background(), badAuth); err != nil {
t.Fatalf("register bad auth: %v", err)
}
if _, err := m.Register(context.Background(), goodAuth); err != nil {
t.Fatalf("register good auth: %v", err)
}
reg := registry.GetGlobalRegistry()
reg.RegisterClient(badAuth.ID, "pool", []*registry.ModelInfo{{ID: alias}})
reg.RegisterClient(goodAuth.ID, "pool", []*registry.ModelInfo{{ID: alias}})
t.Cleanup(func() {
reg.UnregisterClient(badAuth.ID)
reg.UnregisterClient(goodAuth.ID)
})
modelSupportErr := &Error{
HTTPStatus: http.StatusBadRequest,
Message: "invalid_request_error: The requested model is not supported.",
}
for _, upstreamModel := range []string{"qwen3.5-plus", "glm-5"} {
m.MarkResult(context.Background(), Result{
AuthID: badAuth.ID,
Provider: "pool",
Model: upstreamModel,
Success: false,
Error: modelSupportErr,
})
}
resp, err := m.Execute(context.Background(), []string{"pool"}, cliproxyexecutor.Request{Model: alias}, cliproxyexecutor.Options{})
if err != nil {
t.Fatalf("execute error = %v, want success via fallback auth", err)
}
if !strings.HasPrefix(string(resp.Payload), goodAuth.ID+"|") {
t.Fatalf("payload = %q, want auth %q", string(resp.Payload), goodAuth.ID)
}
got := executor.ExecuteCalls()
if len(got) != 1 {
t.Fatalf("execute calls = %v, want only one real execution on fallback auth", got)
}
if !strings.HasPrefix(got[0], goodAuth.ID+"|") {
t.Fatalf("execute call = %q, want fallback auth %q", got[0], goodAuth.ID)
}
}
func TestManagerExecuteStream_OpenAICompatAliasPoolStopsOnInvalidBootstrap(t *testing.T) {
alias := "claude-opus-4.66"
invalidErr := &Error{HTTPStatus: http.StatusBadRequest, Message: "invalid_request_error: malformed payload"}
+53 -5
View File
@@ -293,12 +293,46 @@ func (s *authScheduler) pickMixed(ctx context.Context, providers []string, model
}
cursorKey := strings.Join(normalized, ",") + ":" + modelKey
start := 0
if len(normalized) > 0 {
start = s.mixedCursors[cursorKey] % len(normalized)
weights := make([]int, len(normalized))
segmentStarts := make([]int, len(normalized))
segmentEnds := make([]int, len(normalized))
totalWeight := 0
for providerIndex, shard := range candidateShards {
segmentStarts[providerIndex] = totalWeight
if shard != nil {
weights[providerIndex] = shard.readyCountAtPriorityLocked(false, bestPriority)
}
totalWeight += weights[providerIndex]
segmentEnds[providerIndex] = totalWeight
}
if totalWeight == 0 {
return nil, "", s.mixedUnavailableErrorLocked(normalized, model, tried)
}
startSlot := s.mixedCursors[cursorKey] % totalWeight
startProviderIndex := -1
for providerIndex := range normalized {
if weights[providerIndex] == 0 {
continue
}
if startSlot < segmentEnds[providerIndex] {
startProviderIndex = providerIndex
break
}
}
if startProviderIndex < 0 {
return nil, "", s.mixedUnavailableErrorLocked(normalized, model, tried)
}
slot := startSlot
for offset := 0; offset < len(normalized); offset++ {
providerIndex := (start + offset) % len(normalized)
providerIndex := (startProviderIndex + offset) % len(normalized)
if weights[providerIndex] == 0 {
continue
}
if providerIndex != startProviderIndex {
slot = segmentStarts[providerIndex]
}
providerKey := normalized[providerIndex]
shard := candidateShards[providerIndex]
if shard == nil {
@@ -308,7 +342,7 @@ func (s *authScheduler) pickMixed(ctx context.Context, providers []string, model
if picked == nil {
continue
}
s.mixedCursors[cursorKey] = providerIndex + 1
s.mixedCursors[cursorKey] = slot + 1
return picked, providerKey, nil
}
return nil, "", s.mixedUnavailableErrorLocked(normalized, model, tried)
@@ -704,6 +738,20 @@ func (m *modelScheduler) pickReadyAtPriorityLocked(preferWebsocket bool, priorit
return picked.auth
}
func (m *modelScheduler) readyCountAtPriorityLocked(preferWebsocket bool, priority int) int {
if m == nil {
return 0
}
bucket := m.readyByPriority[priority]
if bucket == nil {
return 0
}
if preferWebsocket && len(bucket.ws.flat) > 0 {
return len(bucket.ws.flat)
}
return len(bucket.all.flat)
}
// unavailableErrorLocked returns the correct unavailable or cooldown error for the shard.
func (m *modelScheduler) unavailableErrorLocked(provider, model string, predicate func(*scheduledAuth) bool) error {
now := time.Now()
+8 -8
View File
@@ -208,7 +208,7 @@ func TestSchedulerPick_CodexWebsocketPrefersWebsocketEnabledSubset(t *testing.T)
}
}
func TestSchedulerPick_MixedProvidersUsesProviderRotationOverReadyCandidates(t *testing.T) {
func TestSchedulerPick_MixedProvidersUsesWeightedProviderRotationOverReadyCandidates(t *testing.T) {
t.Parallel()
scheduler := newSchedulerForTest(
@@ -218,8 +218,8 @@ func TestSchedulerPick_MixedProvidersUsesProviderRotationOverReadyCandidates(t *
&Auth{ID: "claude-a", Provider: "claude"},
)
wantProviders := []string{"gemini", "claude", "gemini", "claude"}
wantIDs := []string{"gemini-a", "claude-a", "gemini-b", "claude-a"}
wantProviders := []string{"gemini", "gemini", "claude", "gemini"}
wantIDs := []string{"gemini-a", "gemini-b", "claude-a", "gemini-a"}
for index := range wantProviders {
got, provider, errPick := scheduler.pickMixed(context.Background(), []string{"gemini", "claude"}, "", cliproxyexecutor.Options{}, nil)
if errPick != nil {
@@ -272,7 +272,7 @@ func TestSchedulerPick_MixedProvidersPrefersHighestPriorityTier(t *testing.T) {
}
}
func TestManager_PickNextMixed_UsesProviderRotationBeforeCredentialRotation(t *testing.T) {
func TestManager_PickNextMixed_UsesWeightedProviderRotationBeforeCredentialRotation(t *testing.T) {
t.Parallel()
manager := NewManager(nil, &RoundRobinSelector{}, nil)
@@ -288,8 +288,8 @@ func TestManager_PickNextMixed_UsesProviderRotationBeforeCredentialRotation(t *t
t.Fatalf("Register(claude-a) error = %v", errRegister)
}
wantProviders := []string{"gemini", "claude", "gemini", "claude"}
wantIDs := []string{"gemini-a", "claude-a", "gemini-b", "claude-a"}
wantProviders := []string{"gemini", "gemini", "claude", "gemini"}
wantIDs := []string{"gemini-a", "gemini-b", "claude-a", "gemini-a"}
for index := range wantProviders {
got, _, provider, errPick := manager.pickNextMixed(context.Background(), []string{"gemini", "claude"}, "", cliproxyexecutor.Options{}, map[string]struct{}{})
if errPick != nil {
@@ -399,8 +399,8 @@ func TestManager_PickNextMixed_UsesSchedulerRotation(t *testing.T) {
t.Fatalf("Register(claude-a) error = %v", errRegister)
}
wantProviders := []string{"gemini", "claude", "gemini", "claude"}
wantIDs := []string{"gemini-a", "claude-a", "gemini-b", "claude-a"}
wantProviders := []string{"gemini", "gemini", "claude", "gemini"}
wantIDs := []string{"gemini-a", "gemini-b", "claude-a", "gemini-a"}
for index := range wantProviders {
got, _, provider, errPick := manager.pickNextMixed(context.Background(), []string{"gemini", "claude"}, "", cliproxyexecutor.Options{}, nil)
if errPick != nil {
+12 -5
View File
@@ -286,10 +286,12 @@ func (s *Service) applyCoreAuthAddOrUpdate(ctx context.Context, auth *coreauth.A
var err error
if existing, ok := s.coreManager.GetByID(auth.ID); ok {
auth.CreatedAt = existing.CreatedAt
auth.LastRefreshedAt = existing.LastRefreshedAt
auth.NextRefreshAfter = existing.NextRefreshAfter
if len(auth.ModelStates) == 0 && len(existing.ModelStates) > 0 {
auth.ModelStates = existing.ModelStates
if !existing.Disabled && existing.Status != coreauth.StatusDisabled && !auth.Disabled && auth.Status != coreauth.StatusDisabled {
auth.LastRefreshedAt = existing.LastRefreshedAt
auth.NextRefreshAfter = existing.NextRefreshAfter
if len(auth.ModelStates) == 0 && len(existing.ModelStates) > 0 {
auth.ModelStates = existing.ModelStates
}
}
op = "update"
_, err = s.coreManager.Update(ctx, auth)
@@ -960,6 +962,10 @@ func (s *Service) registerModelsForAuth(a *coreauth.Auth) {
if modelID == "" {
modelID = m.Name
}
thinking := m.Thinking
if thinking == nil {
thinking = &registry.ThinkingSupport{Levels: []string{"low", "medium", "high"}}
}
ms = append(ms, &ModelInfo{
ID: modelID,
Object: "model",
@@ -967,7 +973,8 @@ func (s *Service) registerModelsForAuth(a *coreauth.Auth) {
OwnedBy: compat.Name,
Type: "openai-compatibility",
DisplayName: modelID,
UserDefined: true,
UserDefined: false,
Thinking: thinking,
})
}
// Register and return
+85
View File
@@ -0,0 +1,85 @@
package cliproxy
import (
"context"
"testing"
"time"
"github.com/router-for-me/CLIProxyAPI/v6/internal/registry"
coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
"github.com/router-for-me/CLIProxyAPI/v6/sdk/config"
)
func TestServiceApplyCoreAuthAddOrUpdate_DeleteReAddDoesNotInheritStaleRuntimeState(t *testing.T) {
service := &Service{
cfg: &config.Config{},
coreManager: coreauth.NewManager(nil, nil, nil),
}
authID := "service-stale-state-auth"
modelID := "stale-model"
lastRefreshedAt := time.Date(2026, time.March, 1, 8, 0, 0, 0, time.UTC)
nextRefreshAfter := lastRefreshedAt.Add(30 * time.Minute)
t.Cleanup(func() {
GlobalModelRegistry().UnregisterClient(authID)
})
service.applyCoreAuthAddOrUpdate(context.Background(), &coreauth.Auth{
ID: authID,
Provider: "claude",
Status: coreauth.StatusActive,
LastRefreshedAt: lastRefreshedAt,
NextRefreshAfter: nextRefreshAfter,
ModelStates: map[string]*coreauth.ModelState{
modelID: {
Quota: coreauth.QuotaState{BackoffLevel: 7},
},
},
})
service.applyCoreAuthRemoval(context.Background(), authID)
disabled, ok := service.coreManager.GetByID(authID)
if !ok || disabled == nil {
t.Fatalf("expected disabled auth after removal")
}
if !disabled.Disabled || disabled.Status != coreauth.StatusDisabled {
t.Fatalf("expected disabled auth after removal, got disabled=%v status=%v", disabled.Disabled, disabled.Status)
}
if disabled.LastRefreshedAt.IsZero() {
t.Fatalf("expected disabled auth to still carry prior LastRefreshedAt for regression setup")
}
if disabled.NextRefreshAfter.IsZero() {
t.Fatalf("expected disabled auth to still carry prior NextRefreshAfter for regression setup")
}
if len(disabled.ModelStates) == 0 {
t.Fatalf("expected disabled auth to still carry prior ModelStates for regression setup")
}
service.applyCoreAuthAddOrUpdate(context.Background(), &coreauth.Auth{
ID: authID,
Provider: "claude",
Status: coreauth.StatusActive,
})
updated, ok := service.coreManager.GetByID(authID)
if !ok || updated == nil {
t.Fatalf("expected re-added auth to be present")
}
if updated.Disabled {
t.Fatalf("expected re-added auth to be active")
}
if !updated.LastRefreshedAt.IsZero() {
t.Fatalf("expected LastRefreshedAt to reset on delete -> re-add, got %v", updated.LastRefreshedAt)
}
if !updated.NextRefreshAfter.IsZero() {
t.Fatalf("expected NextRefreshAfter to reset on delete -> re-add, got %v", updated.NextRefreshAfter)
}
if len(updated.ModelStates) != 0 {
t.Fatalf("expected ModelStates to reset on delete -> re-add, got %d entries", len(updated.ModelStates))
}
if models := registry.GetGlobalRegistry().GetModelsForClient(authID); len(models) == 0 {
t.Fatalf("expected re-added auth to re-register models in global registry")
}
}
+1
View File
@@ -17,6 +17,7 @@ type Record struct {
AuthIndex string
Source string
RequestedAt time.Time
Latency time.Duration
Failed bool
Detail Detail
}
+19 -13
View File
@@ -68,14 +68,18 @@ func Parse(raw string) (Setting, error) {
}
}
func cloneDefaultTransport() *http.Transport {
if transport, ok := http.DefaultTransport.(*http.Transport); ok && transport != nil {
return transport.Clone()
}
return &http.Transport{}
}
// NewDirectTransport returns a transport that bypasses environment proxies.
func NewDirectTransport() *http.Transport {
if transport, ok := http.DefaultTransport.(*http.Transport); ok && transport != nil {
clone := transport.Clone()
clone.Proxy = nil
return clone
}
return &http.Transport{Proxy: nil}
clone := cloneDefaultTransport()
clone.Proxy = nil
return clone
}
// BuildHTTPTransport constructs an HTTP transport for the provided proxy setting.
@@ -102,14 +106,16 @@ func BuildHTTPTransport(raw string) (*http.Transport, Mode, error) {
if errSOCKS5 != nil {
return nil, setting.Mode, fmt.Errorf("create SOCKS5 dialer failed: %w", errSOCKS5)
}
return &http.Transport{
Proxy: nil,
DialContext: func(_ context.Context, network, addr string) (net.Conn, error) {
return dialer.Dial(network, addr)
},
}, setting.Mode, nil
transport := cloneDefaultTransport()
transport.Proxy = nil
transport.DialContext = func(_ context.Context, network, addr string) (net.Conn, error) {
return dialer.Dial(network, addr)
}
return transport, setting.Mode, nil
}
return &http.Transport{Proxy: http.ProxyURL(setting.URL)}, setting.Mode, nil
transport := cloneDefaultTransport()
transport.Proxy = http.ProxyURL(setting.URL)
return transport, setting.Mode, nil
default:
return nil, setting.Mode, nil
}
+50
View File
@@ -5,6 +5,16 @@ import (
"testing"
)
func mustDefaultTransport(t *testing.T) *http.Transport {
t.Helper()
transport, ok := http.DefaultTransport.(*http.Transport)
if !ok || transport == nil {
t.Fatal("http.DefaultTransport is not an *http.Transport")
}
return transport
}
func TestParse(t *testing.T) {
t.Parallel()
@@ -86,4 +96,44 @@ func TestBuildHTTPTransportHTTPProxy(t *testing.T) {
if proxyURL == nil || proxyURL.String() != "http://proxy.example.com:8080" {
t.Fatalf("proxy URL = %v, want http://proxy.example.com:8080", proxyURL)
}
defaultTransport := mustDefaultTransport(t)
if transport.ForceAttemptHTTP2 != defaultTransport.ForceAttemptHTTP2 {
t.Fatalf("ForceAttemptHTTP2 = %v, want %v", transport.ForceAttemptHTTP2, defaultTransport.ForceAttemptHTTP2)
}
if transport.IdleConnTimeout != defaultTransport.IdleConnTimeout {
t.Fatalf("IdleConnTimeout = %v, want %v", transport.IdleConnTimeout, defaultTransport.IdleConnTimeout)
}
if transport.TLSHandshakeTimeout != defaultTransport.TLSHandshakeTimeout {
t.Fatalf("TLSHandshakeTimeout = %v, want %v", transport.TLSHandshakeTimeout, defaultTransport.TLSHandshakeTimeout)
}
}
func TestBuildHTTPTransportSOCKS5ProxyInheritsDefaultTransportSettings(t *testing.T) {
t.Parallel()
transport, mode, errBuild := BuildHTTPTransport("socks5://proxy.example.com:1080")
if errBuild != nil {
t.Fatalf("BuildHTTPTransport returned error: %v", errBuild)
}
if mode != ModeProxy {
t.Fatalf("mode = %d, want %d", mode, ModeProxy)
}
if transport == nil {
t.Fatal("expected transport, got nil")
}
if transport.Proxy != nil {
t.Fatal("expected SOCKS5 transport to bypass http proxy function")
}
defaultTransport := mustDefaultTransport(t)
if transport.ForceAttemptHTTP2 != defaultTransport.ForceAttemptHTTP2 {
t.Fatalf("ForceAttemptHTTP2 = %v, want %v", transport.ForceAttemptHTTP2, defaultTransport.ForceAttemptHTTP2)
}
if transport.IdleConnTimeout != defaultTransport.IdleConnTimeout {
t.Fatalf("IdleConnTimeout = %v, want %v", transport.IdleConnTimeout, defaultTransport.IdleConnTimeout)
}
if transport.TLSHandshakeTimeout != defaultTransport.TLSHandshakeTimeout {
t.Fatalf("TLSHandshakeTimeout = %v, want %v", transport.TLSHandshakeTimeout, defaultTransport.TLSHandshakeTimeout)
}
}
+14 -1
View File
@@ -3,6 +3,10 @@ package translator
import (
"context"
"sync"
log "github.com/sirupsen/logrus"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
)
// Registry manages translation functions across schemas.
@@ -39,7 +43,9 @@ func (r *Registry) Register(from, to Format, request RequestTransform, response
}
// TranslateRequest converts a payload between schemas, returning the original payload
// if no translator is registered.
// if no translator is registered. When falling back to the original payload, the
// "model" field is still updated to match the resolved model name so that
// client-side prefixes (e.g. "copilot/gpt-5-mini") are not leaked upstream.
func (r *Registry) TranslateRequest(from, to Format, model string, rawJSON []byte, stream bool) []byte {
r.mu.RLock()
defer r.mu.RUnlock()
@@ -49,6 +55,13 @@ func (r *Registry) TranslateRequest(from, to Format, model string, rawJSON []byt
return fn(model, rawJSON, stream)
}
}
if model != "" && gjson.GetBytes(rawJSON, "model").String() != model {
if updated, err := sjson.SetBytes(rawJSON, "model", model); err != nil {
log.Warnf("translator: failed to normalize model in request fallback: %v", err)
} else {
return updated
}
}
return rawJSON
}
+92
View File
@@ -0,0 +1,92 @@
package translator
import (
"testing"
"github.com/tidwall/gjson"
)
func TestTranslateRequest_FallbackNormalizesModel(t *testing.T) {
r := NewRegistry()
tests := []struct {
name string
model string
payload string
wantModel string
wantUnchanged bool
}{
{
name: "prefixed model is rewritten",
model: "gpt-5-mini",
payload: `{"model":"copilot/gpt-5-mini","input":"ping"}`,
wantModel: "gpt-5-mini",
},
{
name: "matching model is left unchanged",
model: "gpt-5-mini",
payload: `{"model":"gpt-5-mini","input":"ping"}`,
wantModel: "gpt-5-mini",
wantUnchanged: true,
},
{
name: "empty model leaves payload unchanged",
model: "",
payload: `{"model":"copilot/gpt-5-mini","input":"ping"}`,
wantModel: "copilot/gpt-5-mini",
wantUnchanged: true,
},
{
name: "deeply prefixed model is rewritten",
model: "gpt-5.3-codex",
payload: `{"model":"team/gpt-5.3-codex","stream":true}`,
wantModel: "gpt-5.3-codex",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
input := []byte(tt.payload)
got := r.TranslateRequest(Format("a"), Format("b"), tt.model, input, false)
gotModel := gjson.GetBytes(got, "model").String()
if gotModel != tt.wantModel {
t.Errorf("model = %q, want %q", gotModel, tt.wantModel)
}
if tt.wantUnchanged && string(got) != tt.payload {
t.Errorf("payload was modified when it should not have been:\ngot: %s\nwant: %s", got, tt.payload)
}
// Verify other fields are preserved.
for _, key := range []string{"input", "stream"} {
orig := gjson.Get(tt.payload, key)
if !orig.Exists() {
continue
}
after := gjson.GetBytes(got, key)
if orig.Raw != after.Raw {
t.Errorf("field %q changed: got %s, want %s", key, after.Raw, orig.Raw)
}
}
})
}
}
func TestTranslateRequest_RegisteredTransformTakesPrecedence(t *testing.T) {
r := NewRegistry()
from := Format("openai-response")
to := Format("openai-response")
r.Register(from, to, func(model string, rawJSON []byte, stream bool) []byte {
return []byte(`{"model":"from-transform"}`)
}, ResponseTransform{})
input := []byte(`{"model":"copilot/gpt-5-mini","input":"ping"}`)
got := r.TranslateRequest(from, to, "gpt-5-mini", input, false)
gotModel := gjson.GetBytes(got, "model").String()
if gotModel != "from-transform" {
t.Errorf("expected registered transform to take precedence, got model = %q", gotModel)
}
}