diff --git a/README.fr.md b/README.fr.md index 2e39783c9..0f934ac1b 100644 --- a/README.fr.md +++ b/README.fr.md @@ -90,8 +90,11 @@ vllm serve your-model --port 8000 # vLLM → :8000/v1 python -m sglang.launch_server --model-path your-model --port 30000 ``` -LM Studio (`:1234/v1`), Jan (`:1337/v1`) et LocalAI (`:8080/v1`) fonctionnent de -la même manière. Chargez un modèle avec **au moins une fenêtre de contexte de +LM Studio (`:1234/v1`), Jan (`:1337/v1`), LocalAI (`:8080/v1`) et GPT4All +(`:4891/v1`) fonctionnent de la même manière. Une carte générique **Proxy local +compatible OpenAI** accepte aussi les passerelles de boucle locale authentifiées +comme CLIProxyAPI ; consultez la [configuration sécurisée du proxy](docs/fr/providers-and-models.md#exemple-de-proxy-dabonnement-cliproxyapi). +Chargez un modèle avec **au moins une fenêtre de contexte de 16k jetons** — 8k ne fonctionne qu'avec le niveau Compact, et 4k est trop petit pour le prompt système et les schémas d'outils. WebBrain détecte automatiquement la fenêtre réelle pour llama.cpp, Ollama et LM Studio, et @@ -102,7 +105,7 @@ compacte la conversation à mesure qu'elle se remplit. Il existe aussi un relais **API cloud** — OpenAI, Anthropic Claude, Google Gemini, Azure OpenAI, AWS Bedrock, Mistral, DeepSeek, xAI Grok, MiniMax, Kimi, Qwen, z.ai GLM, Groq, Together, Cloudflare, Nvidia NIM, Hugging Face, Fireworks, OpenRouter et -d'autres. Les Paramètres embarquent **104 cartes de fournisseurs** avec URL de +d'autres. Les Paramètres embarquent **105 cartes de fournisseurs** avec URL de base et modèle par défaut préremplis — voir le [catalogue complet](docs/fr/providers-and-models.md#catalogue-étendu-de-fournisseurs). @@ -195,7 +198,7 @@ WebBrain a le focus. | [Architecture](docs/fr/architecture.md) | Vue d'ensemble, flux d'un tour, sous-systèmes | | [Outils de l'agent](docs/fr/agent-tools.md) | Niveaux, modes et matrice complète | | [Commandes slash](docs/fr/slash-commands.md) | Toutes les commandes et options | -| [Fournisseurs et modèles](docs/fr/providers-and-models.md) | Les 104 cartes, configuration locale, niveaux | +| [Fournisseurs et modèles](docs/fr/providers-and-models.md) | Les 105 cartes, configuration locale, niveaux | | [Compétences](docs/fr/skills.md) | Compétences intégrées, import, outils de compétence | | [Modèle de sécurité](docs/fr/security-model.md) | Permissions, identifiants, frontières de confiance | | [Défense contre l'injection de prompt](docs/fr/prompt-injection-defense.md) | Couches de défense et limites connues | diff --git a/README.md b/README.md index 9419af4d4..d00877a3c 100644 --- a/README.md +++ b/README.md @@ -86,8 +86,11 @@ vllm serve your-model --port 8000 # vLLM → :8000/v1 python -m sglang.launch_server --model-path your-model --port 30000 ``` -LM Studio (`:1234/v1`), Jan (`:1337/v1`), and LocalAI (`:8080/v1`) work the same -way. Load a model with **at least a 16k-token context window** — 8k works only +LM Studio (`:1234/v1`), Jan (`:1337/v1`), LocalAI (`:8080/v1`), and GPT4All +(`:4891/v1`) work the same way. A generic **Local OpenAI-compatible Proxy** card +also supports authenticated loopback gateways such as CLIProxyAPI; see the +[secure subscription proxy setup](docs/providers-and-models.md#subscription-proxy-example-cliproxyapi). +Load a model with **at least a 16k-token context window** — 8k works only with the Compact tier, and 4k is too small for the system prompt plus tool schemas. WebBrain auto-detects the real window for llama.cpp, Ollama, and LM Studio, and auto-compacts the conversation as it fills up. For Ollama, @@ -101,7 +104,7 @@ preview `ollama launch webbrain --model ` handoff. Details: **Cloud APIs** — OpenAI, Anthropic Claude, Google Gemini, Azure OpenAI, AWS Bedrock, Mistral, DeepSeek, xAI Grok, MiniMax, Kimi, Qwen, z.ai GLM, Groq, Together, Cloudflare, Nvidia NIM, Hugging Face, Fireworks, OpenRouter, and more. -Settings ships **104 built-in provider cards** with base URLs and default models +Settings ships **105 built-in provider cards** with base URLs and default models pre-filled — see the [full catalog](docs/providers-and-models.md#extended-provider-catalog). ## Features @@ -185,7 +188,7 @@ Chrome side panel shortcuts work when the WebBrain side panel has focus. | [Architecture](docs/architecture.md) | System overview, turn flow, subsystems | | [Agent tools](docs/agent-tools.md) | Tiers, modes, and the full tool matrix | | [Slash commands](docs/slash-commands.md) | Every command and flag | -| [Providers and models](docs/providers-and-models.md) | All 104 provider cards, local setup, tiers | +| [Providers and models](docs/providers-and-models.md) | All 105 provider cards, local setup, tiers | | [Skills](docs/skills.md) | Bundled skills, importing, skill tools | | [Security model](docs/security-model.md) | Permissions, credentials, trust boundaries | | [Prompt-injection defense](docs/prompt-injection-defense.md) | Defense layers and known gaps | diff --git a/README.zh-CN.md b/README.zh-CN.md index 910601e9e..cd488070d 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -82,7 +82,9 @@ vllm serve your-model --port 8000 # vLLM → :8000/v1 python -m sglang.launch_server --model-path your-model --port 30000 ``` -LM Studio(`:1234/v1`)、Jan(`:1337/v1`)和 LocalAI(`:8080/v1`)用法相同。请加载 +LM Studio(`:1234/v1`)、Jan(`:1337/v1`)、LocalAI(`:8080/v1`)和 GPT4All +(`:4891/v1`)用法相同。通用的**本地 OpenAI 兼容代理**卡片也支持 CLIProxyAPI +等带认证的回环网关;请参阅[安全代理配置](docs/zh-CN/providers-and-models.md#订阅代理示例cliproxyapi)。请加载 **至少具有 16k 令牌上下文窗口**的模型 —— 8k 仅在 Compact 层级下可用,4k 无法容纳 系统提示加工具 schema。WebBrain 会为 llama.cpp、Ollama 和 LM Studio 自动检测真实窗口, 并在对话接近上限时自动压缩。此外还有预览版的 @@ -92,7 +94,7 @@ LM Studio(`:1234/v1`)、Jan(`:1337/v1`)和 LocalAI(`:8080/v1`)用法 **云端 API** —— OpenAI、Anthropic Claude、Google Gemini、Azure OpenAI、AWS Bedrock、 Mistral、DeepSeek、xAI Grok、MiniMax、Kimi、通义千问、z.ai GLM、Groq、Together、 Cloudflare、Nvidia NIM、Hugging Face、Fireworks、OpenRouter 等。设置中内置 -**104 张提供商卡片**,已预填 base URL 和默认模型 —— 参见 +**105 张提供商卡片**,已预填 base URL 和默认模型 —— 参见 [完整目录](docs/zh-CN/providers-and-models.md#扩展提供商目录)。 ## 功能特性 @@ -168,7 +170,7 @@ Chrome 侧边面板快捷键在 WebBrain 侧边面板获得焦点时生效。 | [架构](docs/zh-CN/architecture.md) | 系统概览、轮次流程、子系统 | | [智能体工具](docs/zh-CN/agent-tools.md) | 层级、模式与完整工具矩阵 | | [斜杠命令](docs/zh-CN/slash-commands.md) | 所有命令与参数 | -| [提供商与模型](docs/zh-CN/providers-and-models.md) | 全部 104 张提供商卡片、本地配置、层级 | +| [提供商与模型](docs/zh-CN/providers-and-models.md) | 全部 105 张提供商卡片、本地配置、层级 | | [技能](docs/zh-CN/skills.md) | 内置技能、导入、技能工具 | | [安全模型](docs/zh-CN/security-model.md) | 权限、凭证、信任边界 | | [提示注入防御](docs/zh-CN/prompt-injection-defense.md) | 防御层级与已知缺口 | diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index 640daec31..a25a7ad95 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -63,7 +63,7 @@ The defensible claim is *not* "we're more secure than everyone." It's that the s 1. Is page-derived content structurally isolated from the instruction path? (We do this and test it.) 2. Is the action/automation surface bounded, or does the agent inherit full ambient authority? (Our gap G1/G2/G4 — being closed.) -3. Where does inference run, and what leaves the device? (The selected provider determines this: WebBrain Cloud and bring-your-own cloud providers receive the request context, while local providers keep inference requests on the machine.) +3. Where does inference run, and what leaves the device? (The selected provider determines this: WebBrain Cloud and bring-your-own cloud providers receive the request context; local model runtimes keep inference on the machine, while a local gateway may forward it upstream.) 4. Is there evidence, or just assertions? (Adversarial corpus + ablation, in-repo.) Before making any *comparative* claim about a specific competitor (Edge's AI, OpenAI's browser, the Claude browser, etc.), verify their actual behaviour — don't assert it. The strong, honest line is "here are the dimensions; here's exactly where we stand on each, with tests" and let the comparison speak for itself. diff --git a/docs/fr/THREAT-MODEL.md b/docs/fr/THREAT-MODEL.md index c2f4571d2..d657c2faa 100644 --- a/docs/fr/THREAT-MODEL.md +++ b/docs/fr/THREAT-MODEL.md @@ -63,7 +63,7 @@ L'affirmation défendable n'est *pas* "nous sommes plus sécurisés que tout le 1. Le contenu provenant de la page est-il structurellement isolé du chemin d'instruction ? (Nous le faisons et le testons.) 2. La surface d'action/automatisation est-elle limitée, ou l'agent hérite-t-il de toute l'autorité ambiante ? (Notre lacune G1/G2/G4 — en cours de comblement.) -3. Où l'inférence s'exécute-t-elle, et qu'est-ce qui quitte l'appareil ? (Le fournisseur sélectionné le détermine : WebBrain Cloud et les fournisseurs cloud configurés par l'utilisateur reçoivent le contexte de la requête, tandis que les fournisseurs locaux conservent les requêtes d'inférence sur la machine.) +3. Où l'inférence s'exécute-t-elle, et qu'est-ce qui quitte l'appareil ? (Le fournisseur sélectionné le détermine : WebBrain Cloud et les fournisseurs cloud configurés par l'utilisateur reçoivent le contexte de la requête ; les moteurs de modèles locaux gardent l'inférence sur la machine, tandis qu'une passerelle locale peut la transmettre en amont.) 4. Y a-t-il des preuves, ou juste des affirmations ? (Corpus adversarial + ablation, dans le dépôt.) Avant de faire une affirmation *comparative* sur un concurrent spécifique (Edge AI, navigateur OpenAI, navigateur Claude, etc.), vérifiez leur comportement réel — ne l'affirmez pas. La ligne forte et honnête est "voici les dimensions ; voici exactement où nous en sommes sur chacune, avec des tests" et laissez la comparaison parler d'elle-même. diff --git a/docs/fr/privacy-and-data-flow.md b/docs/fr/privacy-and-data-flow.md index f3a2987d7..24d2d7473 100644 --- a/docs/fr/privacy-and-data-flow.md +++ b/docs/fr/privacy-and-data-flow.md @@ -43,9 +43,13 @@ L'utilisateur choisit son fournisseur dans les Paramètres. Les options incluent Google Gemini, Mistral, DeepSeek, xAI, Groq, OpenRouter, etc. — les requêtes sont envoyées directement au fournisseur avec les identifiants de l'utilisateur et ne sont jamais collectées par WebBrain -- **Fournisseurs locaux** : llama.cpp, Ollama, LM Studio, Jan, vLLM, SGLang, - LocalAI — - les données restent sur la machine de l'utilisateur +- **Moteurs de modèles locaux** : llama.cpp, Ollama, LM Studio, Jan, vLLM, + SGLang, LocalAI et GPT4All — les requêtes d'inférence restent sur la machine + de l'utilisateur +- **Proxy local compatible OpenAI** : WebBrain contacte seulement la passerelle + locale configurée, mais celle-ci peut transmettre le contexte à un compte en + amont. Sa configuration et sa politique de confidentialité déterminent le + trajet des données. Les requêtes vers un modèle local ou une API configurée par l'utilisateur ne sont jamais collectées par WebBrain. Les requêtes WebBrain Cloud sont traitées diff --git a/docs/fr/providers-and-models.md b/docs/fr/providers-and-models.md index 700c21d71..ce06d5722 100644 --- a/docs/fr/providers-and-models.md +++ b/docs/fr/providers-and-models.md @@ -45,6 +45,7 @@ class BaseLLMProvider { | `sglang` | `openai` | local | (modèle chargé) | Oui (activé par défaut) | | `localai` | `openai` | local | (modèle chargé) | Métadonnées auto / surcharge | | `gpt4all` | `openai` | local | (modèle chargé) | Oui (activé par défaut) | +| `local_openai_proxy` | `openai` | local | (requis) | Désactivée / bascule manuelle | | `azure_openai` | `azure_openai` | cloud | (déploiement) | Bascule manuelle | | `aws_bedrock` | `aws_bedrock` | cloud | (ID de modèle) | Non | | `openai` | `openai` | cloud | `gpt-5.6-terra` | Regex nom de modèle | @@ -69,8 +70,8 @@ class BaseLLMProvider { WebBrain ajoute 76 cartes désactivées par défaut depuis l’instantané du catalogue OpenCode au commit -`62e4641235d7847dadc60da37cca8a023dd54fc1`. Avec les 28 cartes existantes, -les Paramètres proposent **104 fournisseurs intégrés**. La liste exacte des +`62e4641235d7847dadc60da37cca8a023dd54fc1`. Avec les 29 cartes existantes, +les Paramètres proposent **105 fournisseurs intégrés**. La liste exacte des identifiants est : `302ai`, `abacus`, `aihubmix`, `alibaba-coding-plan`, @@ -114,8 +115,9 @@ Entrées volontairement exclues : `github-models` (retrait de GitHub Models le ### Fournisseurs Locaux -Sept fournisseurs locaux sont activés par défaut sans clé API requise sauf si le -serveur local a été démarré avec authentification : +Neuf fournisseurs à terminaison locale sont activés par défaut. Les moteurs de +modèles n'exigent pas de clé sauf si le serveur utilise l'authentification ; la +carte proxy générique exige une clé client : - **llama.cpp** : `http://localhost:8080` — exécutez `llama-server -m model.gguf` - **Ollama** : `http://localhost:11434/v1` — `ollama serve`, ou `ollama launch webbrain --model ` @@ -124,6 +126,34 @@ serveur local a été démarré avec authentification : - **vLLM** : `http://localhost:8000/v1` — le serveur compatible OpenAI de vLLM - **SGLang** : `http://localhost:30000/v1` — le serveur compatible OpenAI de SGLang - **LocalAI** : `http://localhost:8080/v1` — le serveur compatible OpenAI de LocalAI +- **GPT4All** : `http://localhost:4891/v1` — le serveur API local de GPT4All +- **Proxy local compatible OpenAI** : `http://127.0.0.1:8317/v1` — passerelle + locale générique authentifiée ; le modèle et la clé API client sont requis + +#### Exemple de proxy d'abonnement (CLIProxyAPI) + +La carte **Proxy local compatible OpenAI** peut joindre une instance +[CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) gérée séparément. +Installez-le via le [guide de démarrage officiel](https://help.router-for.me/introduction/quick-start) +ou compilez-le (`go build -o cli-proxy-api ./cmd/server`), copiez +`config.example.yaml` vers `config.yaml`, puis configurez `host: "127.0.0.1"`, +`port: 8317` et une valeur forte et aléatoire dans `api-keys`. Authentifiez le +compte avec `./cli-proxy-api --config ./config.yaml --codex-login` ou +`--claude-login`. Gemini CLI nécessite le +[plugin officiel](https://github.com/router-for-me/cpa-plugin-gemini-cli) : +activez les plugins de confiance, installez `gemini-cli` depuis le Plugin Store +officiel, redémarrez le proxy, puis utilisez `--geminicli-login` (voir le +[guide de gestion](https://help.router-for.me/management/api#plugins)). Démarrez enfin avec +`./cli-proxy-api --config ./config.yaml`. Dans WebBrain, conservez l'URL +`http://127.0.0.1:8317/v1`, saisissez la même clé, chargez les modèles, +sélectionnez-en un puis testez la connexion. + +Ne publiez pas ce proxy sur le réseau local ou Internet : l'hôte vide par +défaut écoute toutes les interfaces, TLS est désactivé par défaut et une liste +`api-keys` vide autorise les requêtes sans authentification. Ce chemin est +expérimental et communautaire. Le processus est local, mais il peut transmettre +le contexte à un compte en amont ; les identifiants OAuth amont restent dans +CLIProxyAPI. Ollama, llama.cpp, LM Studio et LocalAI utilisent `visionMode: auto` par défaut. WebBrain lit les métadonnées natives du modèle sélectionné avant l'enrichissement et diff --git a/docs/privacy-and-data-flow.md b/docs/privacy-and-data-flow.md index b9b17dd29..3fe4c279b 100644 --- a/docs/privacy-and-data-flow.md +++ b/docs/privacy-and-data-flow.md @@ -41,7 +41,11 @@ The user chooses their provider in Settings. Options include: - **WebBrain Cloud**: requests go through `api.webbrain.one`; selected interactions may be retained and used for evaluation, improvement, fine-tuning, and training while Help Improve WebBrain is enabled - **Bring-your-own cloud providers**: OpenAI, Anthropic, Google Gemini, Mistral, DeepSeek, xAI, Groq, OpenRouter, etc. — requests go directly to the provider using the user's credentials and are never collected by WebBrain -- **Local providers**: llama.cpp, Ollama, LM Studio, Jan, vLLM, SGLang, LocalAI — data stays on the user's machine +- **Local model runtimes**: llama.cpp, Ollama, LM Studio, Jan, vLLM, SGLang, + LocalAI, and GPT4All — inference requests stay on the user's machine +- **Local OpenAI-compatible Proxy**: WebBrain connects only to the configured + local gateway, but the gateway may forward the request context to an upstream + account. Its configuration and privacy policy determine where data goes. Local-model and bring-your-own API requests are never collected by WebBrain. WebBrain Cloud requests are processed and may be retained as described below. diff --git a/docs/providers-and-models.md b/docs/providers-and-models.md index 05360c953..a706302b3 100644 --- a/docs/providers-and-models.md +++ b/docs/providers-and-models.md @@ -45,6 +45,7 @@ class BaseLLMProvider { | `sglang` | `openai` | local | (loaded model) | Yes (default on) | | `localai` | `openai` | local | (loaded model) | Auto metadata / override | | `gpt4all` | `openai` | local | (loaded model) | Yes (default on) | +| `local_openai_proxy` | `openai` | local | (required) | Off / manual toggle | | `azure_openai` | `azure_openai` | cloud | (deployment) | Manual toggle | | `aws_bedrock` | `aws_bedrock` | cloud | (model id) | No | | `openai` | `openai` | cloud | `gpt-5.6-terra` | Model-name regex | @@ -69,8 +70,8 @@ class BaseLLMProvider { WebBrain also ships 76 disabled-by-default provider cards sourced from the OpenCode provider catalog snapshot at commit -`62e4641235d7847dadc60da37cca8a023dd54fc1`. Together with the 28 original -cards, Settings contains **104 built-in providers**. +`62e4641235d7847dadc60da37cca8a023dd54fc1`. Together with the 29 original +cards, Settings contains **105 built-in providers**. | IDs | |---| @@ -144,8 +145,9 @@ duplicate request. ### Local Providers -Seven local providers are enabled by default with no API key needed unless the -local server was started with auth: +Nine local endpoint providers are enabled by default. The model runtimes need no +API key unless the server was started with auth; the generic proxy card requires +a client key so it does not encourage an unauthenticated subscription bridge: - **llama.cpp**: `http://localhost:8080` — runs `llama-server -m model.gguf` - **Ollama**: `http://localhost:11434/v1` — `ollama serve`, or `ollama launch webbrain --model ` @@ -154,6 +156,73 @@ local server was started with auth: - **vLLM**: `http://localhost:8000/v1` — vLLM's OpenAI-compatible server - **SGLang**: `http://localhost:30000/v1` — SGLang's OpenAI-compatible server - **LocalAI**: `http://localhost:8080/v1` — LocalAI's OpenAI-compatible server +- **GPT4All**: `http://localhost:4891/v1` — GPT4All's local API server +- **Local OpenAI-compatible Proxy**: `http://127.0.0.1:8317/v1` — a generic, + authenticated local gateway; the model and proxy client API key are required + +#### Subscription proxy example (CLIProxyAPI) + +The generic **Local OpenAI-compatible Proxy** card can connect WebBrain to a +separately managed [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) +instance. CLIProxyAPI exposes OpenAI-compatible model listing, Chat Completions, +and tool calling while keeping upstream OAuth credentials outside WebBrain. + +Install CLIProxyAPI using its [official quick start](https://help.router-for.me/introduction/quick-start), +or build it from source: + +```bash +git clone https://github.com/router-for-me/CLIProxyAPI.git +cd CLIProxyAPI +go build -o cli-proxy-api ./cmd/server +cp config.example.yaml config.yaml +``` + +Generate a client key (for example, `openssl rand -hex 32`), then edit the +active configuration file before starting the proxy: + +```yaml +host: "127.0.0.1" +port: 8317 +api-keys: + - "replace-with-a-strong-random-local-key" +``` + +Authenticate one or more upstream accounts. The source-build commands are: + +```bash +./cli-proxy-api --config ./config.yaml --codex-login # ChatGPT/Codex account +./cli-proxy-api --config ./config.yaml --claude-login # Claude account +``` + +Gemini CLI OAuth requires CLIProxyAPI v7's +[official Gemini CLI plugin](https://github.com/router-for-me/cpa-plugin-gemini-cli). +Enable trusted plugins, install `gemini-cli` from CLIProxyAPI's official Plugin +Store (the official registry is built in), and restart the proxy before running +it with `--geminicli-login`; see the upstream +[plugin management guide](https://help.router-for.me/management/api#plugins). +Finally, start the source build with +`./cli-proxy-api --config ./config.yaml` (Homebrew/systemd users should restart +the installed service instead). Login flags, plugin packaging, and service paths +can change, so check the linked upstream quick start when commands differ. + +Then open **Settings → Providers → Local OpenAI-compatible Proxy**, keep the +Base URL at `http://127.0.0.1:8317/v1`, enter the same client API key, click +**Load Models**, choose a model, and run **Test Connection**. The proxy's +current official flows cover ChatGPT/Codex and Claude OAuth; Gemini CLI OAuth +is supplied through its official plugin. Available models and quotas still +depend on the connected account. + +Keep this integration on loopback. CLIProxyAPI's empty-host default listens on +all IPv4 and IPv6 interfaces, TLS is off by default, and an empty `api-keys` +list permits unauthenticated requests. Do not expose the proxy to a LAN or the +public internet. This is an experimental, community-supported compatibility +path that may change when provider authentication policies change; official +provider API keys remain the stable option. + +The proxy process is local, but inference is not necessarily local: WebBrain's +request context is forwarded to whichever upstream account the proxy selects. +WebBrain stores only the proxy client key; upstream OAuth credentials remain in +CLIProxyAPI. Ollama, llama.cpp, LM Studio, and LocalAI default to `visionMode: auto`. WebBrain asks the selected server for model capability metadata before enrichment and sends diff --git a/docs/zh-CN/THREAT-MODEL.md b/docs/zh-CN/THREAT-MODEL.md index 80c9f4dbd..7cbebf537 100644 --- a/docs/zh-CN/THREAT-MODEL.md +++ b/docs/zh-CN/THREAT-MODEL.md @@ -63,7 +63,7 @@ 1. 页面内容是否在结构上与指令路径隔离?(我们做到了并且测试了。) 2. 操作/自动化接口是否有限定范围,还是代理继承了完整的环境权限?(我们的差距 G1/G2/G4 —— 正在关闭中。) -3. 推理在哪里运行,什么离开了设备?(这取决于所选提供商:WebBrain Cloud 和用户自带的云提供商会接收请求上下文,而本地提供商的推理请求保留在机器上。) +3. 推理在哪里运行,什么离开了设备?(这取决于所选提供商:WebBrain Cloud 和用户自带的云提供商会接收请求上下文;本地模型运行时把推理保留在机器上,而本地网关可能将其转发到上游。) 4. 是证据还是只是断言?(对抗性测试语料库 + 消融实验,在仓库中。) 在对特定竞争对手(Edge 的 AI、OpenAI 的浏览器、Claude 浏览器等)做出任何*比较*性声明之前,请先验证他们的实际行为——不要断言。强大而诚实的表述是"以下是这些维度;以下是我们在每个维度上的确切位置,附有测试",让比较本身说话。 diff --git a/docs/zh-CN/privacy-and-data-flow.md b/docs/zh-CN/privacy-and-data-flow.md index 35554301a..43034bea7 100644 --- a/docs/zh-CN/privacy-and-data-flow.md +++ b/docs/zh-CN/privacy-and-data-flow.md @@ -28,7 +28,10 @@ - **WebBrain Cloud**:请求会经过 `api.webbrain.one`;“帮助改进 WebBrain”默认启用,在其保持启用期间,部分交互可能会被保留并用于评估、改进、微调和训练 - **用户自带的云提供商**:OpenAI、Anthropic、Google Gemini、Mistral、DeepSeek、xAI、Groq、OpenRouter 等——请求使用用户凭据直接发送给提供商,WebBrain 不会收集这些请求 -- **本地提供商**:llama.cpp、Ollama、LM Studio、Jan、vLLM、SGLang、LocalAI——数据保留在用户的机器上 +- **本地模型运行时**:llama.cpp、Ollama、LM Studio、Jan、vLLM、SGLang、LocalAI + 和 GPT4All——推理请求保留在用户的机器上 +- **本地 OpenAI 兼容代理**:WebBrain 只连接已配置的本地网关,但网关可能把请求上下文 + 转发给上游账户。实际数据去向取决于网关的配置和隐私政策。 本地模型和用户自带 API 的请求不会被 WebBrain 收集。WebBrain Cloud 请求会被处理,并可能按照[英文文档中的详细说明](../privacy-and-data-flow.md#webbrain-cloud-improvement-data)予以保留。 diff --git a/docs/zh-CN/providers-and-models.md b/docs/zh-CN/providers-and-models.md index f4c3a9c18..48409ffb5 100644 --- a/docs/zh-CN/providers-and-models.md +++ b/docs/zh-CN/providers-and-models.md @@ -45,6 +45,7 @@ class BaseLLMProvider { | `sglang` | `openai` | 本地 | (已加载模型) | 是(默认开启) | | `localai` | `openai` | 本地 | (已加载模型) | 自动元数据 / 覆盖 | | `gpt4all` | `openai` | 本地 | (已加载模型) | 是(默认开启) | +| `local_openai_proxy` | `openai` | 本地 | (必填) | 默认关闭 / 手动开关 | | `azure_openai` | `azure_openai` | 云端 | (部署) | 手动开关 | | `aws_bedrock` | `aws_bedrock` | 云端 | (模型 ID) | 否 | | `openai` | `openai` | 云端 | `gpt-5.6-terra` | 模型名正则 | @@ -69,7 +70,7 @@ class BaseLLMProvider { WebBrain 从 OpenCode 提供商目录提交 `62e4641235d7847dadc60da37cca8a023dd54fc1` 的快照中新增了 76 张默认禁用的 -提供商卡片。加上原有 28 张,设置中共有 **104 个内置提供商**。完整 ID +提供商卡片。加上原有 29 张,设置中共有 **105 个内置提供商**。完整 ID 列表如下: `302ai`、`abacus`、`aihubmix`、`alibaba-coding-plan`、 @@ -108,7 +109,8 @@ WebBrain 会直接记录;若服务省略用量,则记录基于字符数的 ### 本地提供商 -七个本地提供商默认启用,无需 API 密钥(除非本地服务器启动时启用了认证): +九个本地端点提供商默认启用。模型运行时无需 API 密钥(除非服务器启用了认证); +通用代理卡片必须填写客户端密钥: - **llama.cpp**:`http://localhost:8080` — 运行 `llama-server -m model.gguf` - **Ollama**:`http://localhost:11434/v1` — `ollama serve`,或 `ollama launch webbrain --model ` @@ -117,6 +119,30 @@ WebBrain 会直接记录;若服务省略用量,则记录基于字符数的 - **vLLM**:`http://localhost:8000/v1` — vLLM 的 OpenAI 兼容服务器 - **SGLang**:`http://localhost:30000/v1` — SGLang 的 OpenAI 兼容服务器 - **LocalAI**:`http://localhost:8080/v1` — LocalAI 的 OpenAI 兼容服务器 +- **GPT4All**:`http://localhost:4891/v1` — GPT4All 本地 API 服务器 +- **本地 OpenAI 兼容代理**:`http://127.0.0.1:8317/v1` — 通用的、带认证的 + 本地网关;模型和代理客户端 API 密钥均为必填 + +#### 订阅代理示例(CLIProxyAPI) + +**本地 OpenAI 兼容代理**卡片可连接单独管理的 +[CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) 实例。按照 +[官方快速入门](https://help.router-for.me/introduction/quick-start)安装,或从源码执行 +`go build -o cli-proxy-api ./cmd/server` 并把 `config.example.yaml` 复制为 +`config.yaml`。先设置 `host: "127.0.0.1"`、`port: 8317`,并在 `api-keys` 中生成 +强随机密钥。使用 `./cli-proxy-api --config ./config.yaml --codex-login` 登录 +ChatGPT/Codex,或用 `--claude-login` 登录 Claude;Gemini CLI 需先安装 +[官方插件](https://github.com/router-for-me/cpa-plugin-gemini-cli):启用可信插件,在 +CLIProxyAPI 官方插件商店安装 `gemini-cli`,重启代理,再使用 `--geminicli-login` +(参见[插件管理说明](https://help.router-for.me/cn/management/api#插件))。最后运行 +`./cli-proxy-api --config ./config.yaml` 启动服务。 +在 WebBrain 中保留 `http://127.0.0.1:8317/v1`,填写同一密钥,加载并选择模型, +最后测试连接。 + +不要把代理暴露到局域网或公网:CLIProxyAPI 的空 host 默认监听所有网络接口,TLS +默认关闭,空 `api-keys` 列表会允许未认证请求。这是一条实验性、由社区支持的兼容 +路径。代理进程在本机运行,但可能把请求上下文转发给上游账户;上游 OAuth 凭据始终 +保留在 CLIProxyAPI 中。 Ollama、llama.cpp、LM Studio 和 LocalAI 默认使用 `visionMode: auto`。WebBrain 在 页面上下文增强前读取所选模型的原生服务器元数据,只有服务器明确报告支持图像输入时才 diff --git a/src/chrome/ARCHITECTURE.md b/src/chrome/ARCHITECTURE.md index 863667ba5..85719b118 100644 --- a/src/chrome/ARCHITECTURE.md +++ b/src/chrome/ARCHITECTURE.md @@ -539,7 +539,7 @@ class BaseProvider { | `LlamaCppProvider` | `localhost:8080/v1/chat/completions` | Enabled by default, configurable | | OpenAI-compatible configs | Provider-specific `/v1` endpoint | Model-name regex or explicit config | -`ProviderManager` seeds WebBrain Cloud, seven local backends, Azure OpenAI, AWS +`ProviderManager` seeds WebBrain Cloud, nine local endpoints, Azure OpenAI, AWS Bedrock, direct cloud providers, and router providers. The canonical current ID and default-model table is maintained in [`docs/providers-and-models.md`](../../docs/providers-and-models.md). diff --git a/src/chrome/icons/providers/SOURCES.md b/src/chrome/icons/providers/SOURCES.md index 80d1eb245..2b4d18dbc 100644 --- a/src/chrome/icons/providers/SOURCES.md +++ b/src/chrome/icons/providers/SOURCES.md @@ -10,6 +10,7 @@ Most brand marks come from [Lobe Icons](https://github.com/lobehub/lobe-icons) | jan | Jan official app icon | | localai | LocalAI official logo (`mudler/LocalAI`) | | gpt4all | Simple mark (not an official GPT4All brand asset) | +| local_openai_proxy | WebBrain-authored generic proxy mark | | sglang | SGLang official `logo_square` (`sgl-project/sglang`) | | webbrain_cloud | WebBrain extension icon | | 302ai, abacus, aihubmix, alibaba-coding-plan, alibaba-coding-plan-cn, azure-cognitive-services, bailing, baseten, berget, cerebras, chutes, clarifai, cloudferro-sherlock, cohere, cortecs, deepinfra, digitalocean, dinference, drun, evroc, fastrouter, friendli, google-vertex, google-vertex-anthropic, helicone, iflowcn, inception, inference, io-net, jiekou, kilo, kimi-for-coding, kuae-cloud-coding-plan, llama, lucidquery, meganova, minimax-cn-coding-plan, minimax-coding-plan, moark, modelscope, morph, nano-gpt, nebius, nova, novita-ai, ollama-cloud, opencode, opencode-go, ovhcloud, perplexity, perplexity-agent, poe, privatemode-ai, qihang-ai, qiniu-ai, requesty, scaleway, siliconflow, siliconflow-cn, stackit, stepfun, submodel, synthetic, tencent-coding-plan, upstage, v0, venice, vercel, vivgrid, vultr, wandb, xiaomi, zai-coding-plan, zenmux, zhipuai, zhipuai-coding-plan | OpenCode provider icon set at commit `62e4641235d7847dadc60da37cca8a023dd54fc1` (MIT); DInference SVG normalized with a `viewBox` | diff --git a/src/chrome/icons/providers/local_openai_proxy.svg b/src/chrome/icons/providers/local_openai_proxy.svg new file mode 100644 index 000000000..309374ec1 --- /dev/null +++ b/src/chrome/icons/providers/local_openai_proxy.svg @@ -0,0 +1,8 @@ + + Local OpenAI-compatible Proxy + + + + + + diff --git a/src/chrome/src/providers/manager.js b/src/chrome/src/providers/manager.js index 7d8a14014..b9874529d 100644 --- a/src/chrome/src/providers/manager.js +++ b/src/chrome/src/providers/manager.js @@ -48,7 +48,7 @@ import { } from './connection-test-assets.js'; const WEBBRAIN_CLOUD_PROVIDER_ID = 'webbrain_cloud'; -const LOCAL_MODEL_LIST_PROVIDER_IDS = ['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all']; +const LOCAL_MODEL_LIST_PROVIDER_IDS = ['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local_openai_proxy']; const WEBBRAIN_CLOUD_CONTEXT_WINDOW = 1000000; const WEBBRAIN_CLOUD_LEGACY_CONTEXT_WINDOW = 256000; const WEBBRAIN_DEVICE_GUID_KEY = 'webbrainDeviceGuid'; @@ -314,6 +314,21 @@ export class ProviderManager { supportsVision: true, enabled: true, }, + local_openai_proxy: { + type: 'openai', + category: 'local', + label: 'Local OpenAI-compatible Proxy', + providerName: 'local-openai-proxy', + baseUrl: 'http://127.0.0.1:8317/v1', + model: '', + requiresModel: true, + contextWindow: 16384, + apiKey: '', + requiresApiKey: true, + supportsAskStreaming: true, + supportsVision: false, + enabled: true, + }, azure_openai: { type: 'azure_openai', category: 'cloud', @@ -711,7 +726,7 @@ export class ProviderManager { /** * Provider category for filter UI. Returns one of: - * 'local' — runs on the user's machine (llama.cpp, ollama, lmstudio, jan, vllm, sglang, localai, gpt4all) + * 'local' — connects through a local endpoint (llama.cpp, ollama, lmstudio, jan, vllm, sglang, localai, gpt4all, local_openai_proxy) * 'cloud' — first-party API endpoint (openai, anthropic, gemini, etc.) * 'router' — multi-model gateways that fan out to many backends (openrouter, cloudflare, nvidia, groq) * Reads `config.category` first; falls back to a per-id table so configs @@ -720,7 +735,7 @@ export class ProviderManager { static categoryFor(id, config) { if (config && config.category) return config.category; if (config?.type === 'llamacpp') return 'local'; - if (['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all'].includes(id)) return 'local'; + if (['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local_openai_proxy'].includes(id)) return 'local'; if (ROUTER_PROVIDER_IDS.includes(id)) return 'router'; return 'cloud'; } @@ -1249,7 +1264,8 @@ export class ProviderManager { /** * Fetch selectable models for local providers. Ollama uses its native - * /api/tags endpoint; llama.cpp, LM Studio, Jan, vLLM, SGLang, and LocalAI use + * /api/tags endpoint; llama.cpp, LM Studio, Jan, vLLM, SGLang, LocalAI, + * GPT4All, and generic local proxies use * OpenAI-compatible /v1/models. */ async listProviderModels(id) { @@ -1258,6 +1274,9 @@ export class ProviderManager { if (!LOCAL_MODEL_LIST_PROVIDER_IDS.includes(id)) { return { ok: false, error: 'Model loading is only supported for local providers' }; } + if (provider.config.requiresApiKey && !String(provider.config.apiKey || '').trim()) { + return { ok: false, error: `${provider.config.label || provider.name} API key is required` }; + } const observedBaseUrl = provider.config.baseUrl; const rawBaseUrl = (observedBaseUrl || '').trim().replace(/\/+$/, ''); diff --git a/src/chrome/src/providers/openai.js b/src/chrome/src/providers/openai.js index 0e6840d2d..56dab64e4 100644 --- a/src/chrome/src/providers/openai.js +++ b/src/chrome/src/providers/openai.js @@ -128,6 +128,9 @@ export class OpenAICompatibleProvider extends BaseLLMProvider { _headers() { const headers = { 'Content-Type': 'application/json' }; const providerName = (this.config.providerName || '').toLowerCase(); + if (this.config.requiresApiKey && !String(this.config.apiKey || '').trim()) { + throw new Error(`${this.config.label || this.name} API key is required.`); + } if (this.config.apiKey) { if (this.config.apiKeyHeader === 'x-goog-api-key') { headers['x-goog-api-key'] = String(this.config.apiKey); diff --git a/src/chrome/src/ui/provider-icons.js b/src/chrome/src/ui/provider-icons.js index e7d6c57c2..9f3795e8a 100644 --- a/src/chrome/src/ui/provider-icons.js +++ b/src/chrome/src/ui/provider-icons.js @@ -20,6 +20,7 @@ export const PROVIDER_ICON_FILES = { sglang: 'sglang.png', localai: 'localai.png', gpt4all: 'gpt4all.svg', + local_openai_proxy: 'local_openai_proxy.svg', azure_openai: 'azure_openai.svg', aws_bedrock: 'aws_bedrock.svg', openai: 'openai.svg', @@ -85,6 +86,7 @@ export const PROVIDER_SHORT_LABELS = { sglang: 'SGLang', localai: 'LocalAI', gpt4all: 'GPT4All', + local_openai_proxy: 'Local Proxy', azure_openai: 'Azure OpenAI', aws_bedrock: 'AWS Bedrock', openai: 'OpenAI', diff --git a/src/chrome/src/ui/settings.js b/src/chrome/src/ui/settings.js index c77b9b07d..7022e7de3 100644 --- a/src/chrome/src/ui/settings.js +++ b/src/chrome/src/ui/settings.js @@ -2118,7 +2118,7 @@ function providerApiKeyWarning(id, config) { const input = document.querySelector(`input[data-provider="${id}"][data-key="apiKey"]`); if (!input) return ''; const apiKey = String(config.apiKey || '').trim(); - const keyIsOptional = providersData[id]?.category === 'local'; + const keyIsOptional = providersData[id]?.category === 'local' && config.requiresApiKey !== true; const looksInvalid = apiKey ? apiKey.length < MIN_API_KEY_LENGTH : !keyIsOptional; input.setAttribute('aria-invalid', looksInvalid ? 'true' : 'false'); return looksInvalid ? t('st.providers.api_key_warning') : ''; @@ -2405,6 +2405,16 @@ function renderProviders() { PROMPT_TIER_FIELD, ], }, + local_openai_proxy: { + fields: [ + { key: 'baseUrl', labelKey: 'st.provider.field.server_url', type: 'text', placeholder: 'http://127.0.0.1:8317/v1' }, + { key: 'apiKey', labelKey: 'st.provider.field.api_key', type: 'password', placeholder: 'required — use the proxy client API key' }, + { key: 'model', labelKey: 'st.provider.field.model', type: 'text', placeholder: 'model exposed by the proxy' }, + CONTEXT_WINDOW_FIELD, + { key: 'supportsVision', labelKey: 'st.provider.field.supports_vision', type: 'checkbox' }, + PROMPT_TIER_FIELD, + ], + }, azure_openai: { fields: [ { key: 'baseUrl', labelKey: 'st.provider.field.api_base_url', type: 'text', placeholder: 'https://{resource}.openai.azure.com' }, @@ -2734,7 +2744,7 @@ function renderProviders() { `; } else { - const localModelProviders = ['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all']; + const localModelProviders = ['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local_openai_proxy']; const canLoadModels = localModelProviders.includes(id) && field.key === 'model'; const listAttr = canLoadModels ? `list="models-${id}"` : ''; const datalistHTML = canLoadModels ? `` : ''; diff --git a/src/chrome/src/ui/sidepanel.js b/src/chrome/src/ui/sidepanel.js index e7a876a04..2db430801 100644 --- a/src/chrome/src/ui/sidepanel.js +++ b/src/chrome/src/ui/sidepanel.js @@ -224,7 +224,7 @@ const pinCoachmarkDismissed = (async function initPinCoachmark() { const localModels = document.getElementById('ob-local-models'); const localModelList = document.getElementById('ob-local-model-list'); const totalSteps = steps.length; - const LOCAL_PROVIDER_ORDER = ['jan', 'lmstudio', 'ollama', 'llamacpp', 'vllm', 'sglang', 'localai', 'gpt4all']; + const LOCAL_PROVIDER_ORDER = ['local_openai_proxy', 'jan', 'lmstudio', 'ollama', 'llamacpp', 'vllm', 'sglang', 'localai', 'gpt4all']; let current = 0; let localScanStarted = false; let localModelChoices = []; diff --git a/src/firefox/ARCHITECTURE.md b/src/firefox/ARCHITECTURE.md index 6397efb2f..565435f7f 100644 --- a/src/firefox/ARCHITECTURE.md +++ b/src/firefox/ARCHITECTURE.md @@ -442,7 +442,7 @@ Plus the legacy handlers: `read_page`, `click`, `type_text`, `press_keys`, `scro ## Provider System Identical to Chrome at the provider-class and configuration layer: -WebBrain Cloud, seven local backends, Azure OpenAI, AWS Bedrock, Anthropic, and +WebBrain Cloud, nine local endpoints, Azure OpenAI, AWS Bedrock, Anthropic, and the current direct-cloud/router OpenAI-compatible configs use the same message format and conversion logic. The canonical current ID and default-model table is maintained in diff --git a/src/firefox/icons/providers/SOURCES.md b/src/firefox/icons/providers/SOURCES.md index 80d1eb245..2b4d18dbc 100644 --- a/src/firefox/icons/providers/SOURCES.md +++ b/src/firefox/icons/providers/SOURCES.md @@ -10,6 +10,7 @@ Most brand marks come from [Lobe Icons](https://github.com/lobehub/lobe-icons) | jan | Jan official app icon | | localai | LocalAI official logo (`mudler/LocalAI`) | | gpt4all | Simple mark (not an official GPT4All brand asset) | +| local_openai_proxy | WebBrain-authored generic proxy mark | | sglang | SGLang official `logo_square` (`sgl-project/sglang`) | | webbrain_cloud | WebBrain extension icon | | 302ai, abacus, aihubmix, alibaba-coding-plan, alibaba-coding-plan-cn, azure-cognitive-services, bailing, baseten, berget, cerebras, chutes, clarifai, cloudferro-sherlock, cohere, cortecs, deepinfra, digitalocean, dinference, drun, evroc, fastrouter, friendli, google-vertex, google-vertex-anthropic, helicone, iflowcn, inception, inference, io-net, jiekou, kilo, kimi-for-coding, kuae-cloud-coding-plan, llama, lucidquery, meganova, minimax-cn-coding-plan, minimax-coding-plan, moark, modelscope, morph, nano-gpt, nebius, nova, novita-ai, ollama-cloud, opencode, opencode-go, ovhcloud, perplexity, perplexity-agent, poe, privatemode-ai, qihang-ai, qiniu-ai, requesty, scaleway, siliconflow, siliconflow-cn, stackit, stepfun, submodel, synthetic, tencent-coding-plan, upstage, v0, venice, vercel, vivgrid, vultr, wandb, xiaomi, zai-coding-plan, zenmux, zhipuai, zhipuai-coding-plan | OpenCode provider icon set at commit `62e4641235d7847dadc60da37cca8a023dd54fc1` (MIT); DInference SVG normalized with a `viewBox` | diff --git a/src/firefox/icons/providers/local_openai_proxy.svg b/src/firefox/icons/providers/local_openai_proxy.svg new file mode 100644 index 000000000..309374ec1 --- /dev/null +++ b/src/firefox/icons/providers/local_openai_proxy.svg @@ -0,0 +1,8 @@ + + Local OpenAI-compatible Proxy + + + + + + diff --git a/src/firefox/src/providers/manager.js b/src/firefox/src/providers/manager.js index b335d4932..e581675b6 100644 --- a/src/firefox/src/providers/manager.js +++ b/src/firefox/src/providers/manager.js @@ -42,7 +42,7 @@ import { } from './connection-test-assets.js'; const WEBBRAIN_CLOUD_PROVIDER_ID = 'webbrain_cloud'; -const LOCAL_MODEL_LIST_PROVIDER_IDS = ['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all']; +const LOCAL_MODEL_LIST_PROVIDER_IDS = ['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local_openai_proxy']; const WEBBRAIN_CLOUD_CONTEXT_WINDOW = 1000000; const WEBBRAIN_CLOUD_LEGACY_CONTEXT_WINDOW = 256000; const WEBBRAIN_DEVICE_GUID_KEY = 'webbrainDeviceGuid'; @@ -302,6 +302,21 @@ export class ProviderManager { supportsVision: true, enabled: true, }, + local_openai_proxy: { + type: 'openai', + category: 'local', + label: 'Local OpenAI-compatible Proxy', + providerName: 'local-openai-proxy', + baseUrl: 'http://127.0.0.1:8317/v1', + model: '', + requiresModel: true, + contextWindow: 16384, + apiKey: '', + requiresApiKey: true, + supportsAskStreaming: true, + supportsVision: false, + enabled: true, + }, azure_openai: { type: 'azure_openai', category: 'cloud', @@ -695,7 +710,7 @@ export class ProviderManager { static categoryFor(id, config) { if (config && config.category) return config.category; if (config?.type === 'llamacpp') return 'local'; - if (['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all'].includes(id)) return 'local'; + if (['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local_openai_proxy'].includes(id)) return 'local'; if (ROUTER_PROVIDER_IDS.includes(id)) return 'router'; return 'cloud'; } @@ -1228,7 +1243,8 @@ export class ProviderManager { /** * Fetch selectable models for local providers. Ollama uses its native - * /api/tags endpoint; llama.cpp, LM Studio, Jan, vLLM, SGLang, and LocalAI use + * /api/tags endpoint; llama.cpp, LM Studio, Jan, vLLM, SGLang, LocalAI, + * GPT4All, and generic local proxies use * OpenAI-compatible /v1/models. */ async listProviderModels(id) { @@ -1237,6 +1253,9 @@ export class ProviderManager { if (!LOCAL_MODEL_LIST_PROVIDER_IDS.includes(id)) { return { ok: false, error: 'Model loading is only supported for local providers' }; } + if (provider.config.requiresApiKey && !String(provider.config.apiKey || '').trim()) { + return { ok: false, error: `${provider.config.label || provider.name} API key is required` }; + } const observedBaseUrl = provider.config.baseUrl; const rawBaseUrl = (observedBaseUrl || '').trim().replace(/\/+$/, ''); diff --git a/src/firefox/src/providers/openai.js b/src/firefox/src/providers/openai.js index 01fb1830a..95cb8dca6 100644 --- a/src/firefox/src/providers/openai.js +++ b/src/firefox/src/providers/openai.js @@ -128,6 +128,9 @@ export class OpenAICompatibleProvider extends BaseLLMProvider { _headers() { const headers = { 'Content-Type': 'application/json' }; const providerName = (this.config.providerName || '').toLowerCase(); + if (this.config.requiresApiKey && !String(this.config.apiKey || '').trim()) { + throw new Error(`${this.config.label || this.name} API key is required.`); + } if (this.config.apiKey) { if (this.config.apiKeyHeader === 'x-goog-api-key') { headers['x-goog-api-key'] = String(this.config.apiKey); diff --git a/src/firefox/src/ui/provider-icons.js b/src/firefox/src/ui/provider-icons.js index e7d6c57c2..9f3795e8a 100644 --- a/src/firefox/src/ui/provider-icons.js +++ b/src/firefox/src/ui/provider-icons.js @@ -20,6 +20,7 @@ export const PROVIDER_ICON_FILES = { sglang: 'sglang.png', localai: 'localai.png', gpt4all: 'gpt4all.svg', + local_openai_proxy: 'local_openai_proxy.svg', azure_openai: 'azure_openai.svg', aws_bedrock: 'aws_bedrock.svg', openai: 'openai.svg', @@ -85,6 +86,7 @@ export const PROVIDER_SHORT_LABELS = { sglang: 'SGLang', localai: 'LocalAI', gpt4all: 'GPT4All', + local_openai_proxy: 'Local Proxy', azure_openai: 'Azure OpenAI', aws_bedrock: 'AWS Bedrock', openai: 'OpenAI', diff --git a/src/firefox/src/ui/settings.js b/src/firefox/src/ui/settings.js index c2ee47726..5501f9038 100644 --- a/src/firefox/src/ui/settings.js +++ b/src/firefox/src/ui/settings.js @@ -1857,7 +1857,7 @@ function providerApiKeyWarning(id, config) { const input = document.querySelector(`input[data-provider="${id}"][data-key="apiKey"]`); if (!input) return ''; const apiKey = String(config.apiKey || '').trim(); - const keyIsOptional = providersData[id]?.category === 'local'; + const keyIsOptional = providersData[id]?.category === 'local' && config.requiresApiKey !== true; const looksInvalid = apiKey ? apiKey.length < MIN_API_KEY_LENGTH : !keyIsOptional; input.setAttribute('aria-invalid', looksInvalid ? 'true' : 'false'); return looksInvalid ? t('st.providers.api_key_warning') : ''; @@ -2144,6 +2144,16 @@ function renderProviders() { PROMPT_TIER_FIELD, ], }, + local_openai_proxy: { + fields: [ + { key: 'baseUrl', labelKey: 'st.provider.field.server_url', type: 'text', placeholder: 'http://127.0.0.1:8317/v1' }, + { key: 'apiKey', labelKey: 'st.provider.field.api_key', type: 'password', placeholder: 'required — use the proxy client API key' }, + { key: 'model', labelKey: 'st.provider.field.model', type: 'text', placeholder: 'model exposed by the proxy' }, + CONTEXT_WINDOW_FIELD, + { key: 'supportsVision', labelKey: 'st.provider.field.supports_vision', type: 'checkbox' }, + PROMPT_TIER_FIELD, + ], + }, azure_openai: { fields: [ { key: 'baseUrl', labelKey: 'st.provider.field.api_base_url', type: 'text', placeholder: 'https://{resource}.openai.azure.com' }, @@ -2468,7 +2478,7 @@ function renderProviders() { `; } else { - const localModelProviders = ['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all']; + const localModelProviders = ['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local_openai_proxy']; const canLoadModels = localModelProviders.includes(id) && field.key === 'model'; const listAttr = canLoadModels ? `list="models-${id}"` : ''; const datalistHTML = canLoadModels ? `` : ''; diff --git a/src/firefox/src/ui/sidepanel.js b/src/firefox/src/ui/sidepanel.js index 5fe57f59b..73b096ef8 100644 --- a/src/firefox/src/ui/sidepanel.js +++ b/src/firefox/src/ui/sidepanel.js @@ -106,7 +106,7 @@ if (globalThis.browser?.storage?.onChanged) { const localModels = document.getElementById('ob-local-models'); const localModelList = document.getElementById('ob-local-model-list'); const totalSteps = steps.length; - const LOCAL_PROVIDER_ORDER = ['jan', 'lmstudio', 'ollama', 'llamacpp', 'vllm', 'sglang', 'localai', 'gpt4all']; + const LOCAL_PROVIDER_ORDER = ['local_openai_proxy', 'jan', 'lmstudio', 'ollama', 'llamacpp', 'vllm', 'sglang', 'localai', 'gpt4all']; let current = 0; let localScanStarted = false; let localModelChoices = []; diff --git a/test/run.js b/test/run.js index 3e8a701a9..b6b43d0ba 100644 --- a/test/run.js +++ b/test/run.js @@ -26250,8 +26250,8 @@ test('settings warns on missing or short API keys and shows the Ollama localhost assert.match(settings, /const MIN_API_KEY_LENGTH = 12;/, `${label}: conservative API-key minimum missing`); assert.match( settings, - /function providerApiKeyWarning\(id, config\) \{[\s\S]*?data-key="apiKey"[\s\S]*?const keyIsOptional = providersData\[id\]\?\.category === 'local';[\s\S]*?apiKey\.length < MIN_API_KEY_LENGTH[\s\S]*?aria-invalid[\s\S]*?st\.providers\.api_key_warning/, - `${label}: API-key warning should cover required empty keys and short non-empty keys while allowing empty local auth`, + /function providerApiKeyWarning\(id, config\) \{[\s\S]*?data-key="apiKey"[\s\S]*?const keyIsOptional = providersData\[id\]\?\.category === 'local' && config\.requiresApiKey !== true;[\s\S]*?apiKey\.length < MIN_API_KEY_LENGTH[\s\S]*?aria-invalid[\s\S]*?st\.providers\.api_key_warning/, + `${label}: API-key warning should allow ordinary empty local auth while enforcing authenticated local proxies`, ); assert.match( settings, @@ -40145,7 +40145,7 @@ console.log('\nprovider categorization'); test('categoryFor: local family', () => { for (const PM of [ProviderManagerCh, ProviderManagerFx]) { - for (const id of ['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all']) { + for (const id of ['llamacpp', 'ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local_openai_proxy']) { assert.equal(PM.categoryFor(id, { type: id === 'llamacpp' ? 'llamacpp' : 'openai' }), 'local'); } assert.equal(PM.categoryFor('custom_llama_cpp', { type: 'llamacpp' }), 'local'); @@ -40443,7 +40443,7 @@ test('tri-state local vision controls and pre-enrichment preparation stay mirror test('inferContextWindow: model-aware cloud/router defaults and local 16k fallback', () => { for (const infer of [inferContextWindowCh, inferContextWindowFx]) { - for (const providerName of ['lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all']) { + for (const providerName of ['lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local-openai-proxy']) { assert.equal(infer({ category: 'local', providerName, model: 'qwen3.7-plus' }), 16384); } for (const model of ['gpt-5.6', 'gpt-5.6-sol', 'gpt-5.6-terra', 'gpt-5.6-luna']) { @@ -41458,7 +41458,7 @@ test('listProviderModels sends saved API keys for auth-enabled OpenAI-compatible try { for (const PM of [ProviderManagerCh, ProviderManagerFx]) { - for (const id of ['jan', 'vllm', 'sglang', 'localai', 'gpt4all']) { + for (const id of ['jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local_openai_proxy']) { const mgr = new PM(); const config = { ...mgr._defaultConfigs()[id], @@ -42185,7 +42185,7 @@ test('extended provider catalog is complete, mirrored, safe, and excluded-provid ['firefox', ProviderManagerFx, 'src/firefox'], ]) { const defaults = new PM()._defaultConfigs(); - assert.equal(Object.keys(defaults).length, 104, `${label}: expected 28 original + 76 new providers`); + assert.equal(Object.keys(defaults).length, 105, `${label}: expected 29 original + 76 catalog providers`); for (const id of expectedIds) { const config = defaults[id]; assert.ok(config, `${label}: missing ${id}`); @@ -42984,16 +42984,106 @@ test('_defaultConfigs: new offline providers present and enabled by default', () for (const PM of [ProviderManagerCh, ProviderManagerFx]) { const mgr = new PM(); const defaults = mgr._defaultConfigs(); - for (const id of ['jan', 'vllm', 'sglang', 'localai', 'gpt4all']) { + for (const id of ['jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local_openai_proxy']) { assert.ok(defaults[id], `${PM.name}: missing default config for ${id}`); assert.equal(defaults[id].type, 'openai', `${PM.name}: ${id} should use OpenAI-compatible provider`); assert.equal(defaults[id].category, 'local', `${PM.name}: ${id} should be local`); assert.equal(defaults[id].enabled, true, `${PM.name}: ${id} should default to enabled`); - assert.ok(defaults[id].baseUrl?.startsWith('http://localhost:'), `${PM.name}: ${id} should use localhost by default`); + assert.match(defaults[id].baseUrl || '', /^http:\/\/(?:localhost|127\.0\.0\.1):/, `${PM.name}: ${id} should use a loopback host by default`); } } }); +test('_defaultConfigs: generic local OpenAI-compatible proxy is safe and configurable', () => { + for (const [label, PM, settingsRel] of [ + ['chrome', ProviderManagerCh, 'src/chrome/src/ui/settings.js'], + ['firefox', ProviderManagerFx, 'src/firefox/src/ui/settings.js'], + ]) { + const config = new PM()._defaultConfigs().local_openai_proxy; + assert.ok(config, `${label}: missing generic local proxy provider`); + assert.equal(config.type, 'openai', `${label}: proxy should reuse the OpenAI-compatible adapter`); + assert.equal(config.category, 'local', `${label}: proxy endpoint should appear under Local`); + assert.equal(config.label, 'Local OpenAI-compatible Proxy'); + assert.equal(config.providerName, 'local-openai-proxy'); + assert.equal(config.model, '', `${label}: users must choose a proxy-exposed model`); + assert.equal(config.requiresModel, true, `${label}: proxy requests need an explicit model`); + assert.equal(config.contextWindow, 16384, `${label}: proxy should inherit the conservative local context default`); + assert.equal(config.apiKey, '', `${label}: no shared credential should ship`); + assert.equal(config.requiresApiKey, true, `${label}: proxy authentication must be required`); + assert.equal(config.supportsAskStreaming, true, `${label}: compatible proxy should use Ask streaming with fallback`); + assert.equal(config.supportsVision, false, `${label}: unknown proxy models should default to text-only`); + assert.equal(config.enabled, true, `${label}: provider card should be available on upgrade`); + + const settings = fs.readFileSync(path.join(ROOT, settingsRel), 'utf8'); + const block = settings.slice( + settings.indexOf('local_openai_proxy: {'), + settings.indexOf('azure_openai: {'), + ); + assert.match(block, /key: 'baseUrl'/, `${label}: proxy Base URL field missing`); + assert.match(block, /key: 'apiKey'/, `${label}: proxy API key field missing`); + assert.match(block, /key: 'model'/, `${label}: proxy model field missing`); + assert.match(block, /CONTEXT_WINDOW_FIELD/, `${label}: proxy context field missing`); + assert.match(block, /key: 'supportsVision'/, `${label}: proxy vision override missing`); + assert.match(block, /PROMPT_TIER_FIELD/, `${label}: proxy prompt tier field missing`); + } +}); + +test('generic local proxy requires authentication and supports non-streaming chat', async () => { + const originalFetch = globalThis.fetch; + try { + for (const [label, PM] of [ + ['chrome', ProviderManagerCh], + ['firefox', ProviderManagerFx], + ]) { + const manager = new PM(); + const defaults = manager._defaultConfigs().local_openai_proxy; + const missingKeyProvider = manager._createProvider('local_openai_proxy', { + ...defaults, + model: 'test-model', + }); + let fetchCount = 0; + globalThis.fetch = async () => { + fetchCount += 1; + throw new Error('unauthenticated request should not be sent'); + }; + await assert.rejects( + missingKeyProvider.chat([{ role: 'user', content: 'hello' }]), + /Local OpenAI-compatible Proxy API key is required/, + `${label}: chat should fail closed without the proxy client key`, + ); + manager.providers.set('local_openai_proxy', missingKeyProvider); + assert.deepEqual( + await manager.listProviderModels('local_openai_proxy'), + { ok: false, error: 'Local OpenAI-compatible Proxy API key is required' }, + `${label}: model listing should fail closed without the proxy client key`, + ); + assert.equal(fetchCount, 0, `${label}: missing-key calls must stop before fetch`); + + let request = null; + globalThis.fetch = async (url, init) => { + request = { url: String(url), headers: init.headers, body: JSON.parse(init.body) }; + return new Response(JSON.stringify({ + choices: [{ message: { content: 'proxy answer' } }], + usage: { prompt_tokens: 2, completion_tokens: 1, total_tokens: 3 }, + }), { status: 200, headers: { 'Content-Type': 'application/json' } }); + }; + const provider = manager._createProvider('local_openai_proxy', { + ...defaults, + apiKey: 'strong-local-proxy-key', + model: 'account-model', + }); + const result = await provider.chat([{ role: 'user', content: 'hello' }]); + assert.equal(result.content, 'proxy answer', `${label}: chat response mismatch`); + assert.equal(request.url, 'http://127.0.0.1:8317/v1/chat/completions'); + assert.equal(request.headers.Authorization, 'Bearer strong-local-proxy-key'); + assert.equal(request.body.model, 'account-model'); + assert.deepEqual(request.body.messages, [{ role: 'user', content: 'hello' }]); + } + } finally { + globalThis.fetch = originalFetch; + } +}); + test('_defaultConfigs: OpenRouter defaults to openrouter/free and migrates legacy default', () => { for (const PM of [ProviderManagerCh, ProviderManagerFx]) { const mgr = new PM(); @@ -43326,6 +43416,7 @@ test('documented built-in providers opt into interactive Ask streaming', () => { 'sglang', 'localai', 'gpt4all', + 'local_openai_proxy', 'azure_openai', 'anthropic', 'gemini', @@ -43345,7 +43436,10 @@ test('documented built-in providers opt into interactive Ask streaming', () => { for (const id of enabledIds) { assert.equal(defaults[id].supportsAskStreaming, true, `${PM.name}/${id}: default capability missing`); assert.equal( - manager._createProvider(id, defaults[id])._supportsInteractiveAskStreaming(), + manager._createProvider(id, { + ...defaults[id], + model: defaults[id].model || 'test-model', + })._supportsInteractiveAskStreaming(), true, `${PM.name}/${id}: provider should expose interactive Ask streaming`, ); @@ -43903,6 +43997,7 @@ test('OpenAI-compatible Ask providers consume text, tool, usage, and DONE fixtur 'sglang', 'localai', 'gpt4all', + 'local_openai_proxy', 'gemini', 'mistral', 'deepseek', @@ -43928,7 +44023,11 @@ test('OpenAI-compatible Ask providers consume text, tool, usage, and DONE fixtur `data: ${JSON.stringify({ choices: [], usage: { prompt_tokens: 5, completion_tokens: 3, total_tokens: 8 } })}\n\n`, 'data: [DONE]\n\n', ].join(''), { status: 200, headers: { 'Content-Type': 'text/event-stream' } }); - const provider = manager._createProvider(id, defaults[id]); + const provider = manager._createProvider(id, { + ...defaults[id], + model: defaults[id].model || 'test-model', + apiKey: defaults[id].requiresApiKey ? 'strong-test-proxy-key' : defaults[id].apiKey, + }); const chunks = []; for await (const chunk of provider.chatStream([{ role: 'user', content: 'hello' }])) chunks.push(chunk); assert.deepEqual(chunks, [ @@ -45167,6 +45266,7 @@ test('OpenAI-compatible local streams do not request usage metadata', () => { { category: 'local', providerName: 'sglang' }, { category: 'local', providerName: 'localai' }, { category: 'local', providerName: 'gpt4all' }, + { category: 'local', providerName: 'local-openai-proxy' }, { category: 'local', providerName: 'openai' }, ]) { const provider = new Provider(config); @@ -45179,7 +45279,7 @@ test('OpenAI-compatible local streams do not request usage metadata', () => { test('OpenAI-compatible local providers always use legacy request token fields', () => { for (const Provider of [OpenAIProviderCh, OpenAIProviderFx]) { - for (const providerName of ['ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all']) { + for (const providerName of ['ollama', 'lmstudio', 'jan', 'vllm', 'sglang', 'localai', 'gpt4all', 'local-openai-proxy']) { const provider = new Provider({ category: 'local', providerName,