diff --git a/ROADMAP.md b/ROADMAP.md index fe4c3a7d48..39f8cd4748 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -8014,3 +8014,7 @@ Original filing (2026-04-18): the session emitted `SessionStart hook (completed) 833. **DONE — Native Ollama provider support via `OLLAMA_HOST` env var** — PR #3213 showed the routing gap: local model names like `qwen3:8b` don't match any provider prefix and `OPENAI_BASE_URL` was the only workaround. Ollama users had to set two env vars (`OPENAI_BASE_URL` + `OPENAI_API_KEY`) and unset Anthropic vars. Fix: (1) `OLLAMA_HOST` env var takes priority over all other routing; when set, all models route to the local OpenAI-compatible endpoint; (2) no API key required — placeholder token used for Authorization header; (3) model names with colons/dots bypass strict `provider/model` syntax validation when `OLLAMA_HOST` is set; (4) `detect_provider_kind()` checks `OLLAMA_HOST` first in the routing cascade; (5) `ProviderClient` dispatch uses `from_ollama_env()` when `OLLAMA_HOST` is set; (6) `OLLAMA_CONFIG` constant added to `openai_compat.rs`; (7) `from_ollama_env()` method constructs client from env with no auth requirement; (8) updated `USAGE.md` and `docs/local-openai-compatible-providers.md` to recommend `OLLAMA_HOST` as the preferred env var; (9) added `test_ollama_host_bypasses_model_validation` unit test. Supersedes PR #3213 (which had a duplicate `if let` bug in `mod.rs`). [SCOPE: claw-code] **Verification.** `cargo fmt --manifest-path rust/Cargo.toml --all -- --check`; `cargo test --manifest-path rust/Cargo.toml -p api -p rusty-claude-cli --bin claw`; `cargo test --manifest-path rust/Cargo.toml -p rusty-claude-cli --test output_format_contract`; dogfooded against mock OpenAI-compatible server with `OLLAMA_HOST=http://127.0.0.1:11434` — `claw --model "qwen3:8b" prompt "Reply exactly: HELLO_WORLD_123"` returned expected output; `claw --output-format json --model "qwen3:8b" prompt "test"` returned parseable JSON; `claw --output-format json doctor` returned `status: warn` (expected, no Anthropic creds); `claw --output-format json --model "qwen3:8b" status` returned `status: ok, model: qwen3:8b`; empty model still rejected with typed error. + +834. **DONE — Local-model syntax bypass only covered Ollama-shaped tags, not bare LM Studio/llama.cpp aliases** — follow-up to #833/PR #3213. `is_local_openai_model_syntax()` waived strict `provider/model` syntax under `OPENAI_BASE_URL` only when the model name contained a `.` or `:` (Ollama-tag-shaped). A local server registering a model under a bare alias with none of those characters — for example an LM Studio custom name or a llama.cpp `--alias` value like `mistral` — still failed CLI-parse-time validation unless the user discovered the `local/` prefix workaround. Fix: (1) `is_local_openai_compatible_base_url()` in `openai_compat.rs`, already used to waive the API-key requirement for loopback/private-network base URLs, is now `pub` and re-exported from the `api` crate root; (2) `is_local_openai_model_syntax()` in `rusty-claude-cli`'s `main.rs` calls it against the configured `OPENAI_BASE_URL` and accepts any non-empty model name when the host is local/private, matching the existing `OLLAMA_HOST` bypass; (3) remote/hosted `OPENAI_BASE_URL` gateways keep the stricter dot/colon-tag heuristic so a mistyped bare name doesn't silently route to a paid hosted API; (4) `local/` remains the explicit escape hatch for slash-containing IDs regardless of host; (5) added an LM Studio section and updated routing notes in `docs/local-openai-compatible-providers.md`. [SCOPE: claw-code] + + **Verification.** `scripts/fmt.sh --check`; `cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings`; `cargo test --manifest-path rust/Cargo.toml --workspace`; new tests `local_openai_base_url_bypasses_bare_model_names`, `private_network_openai_base_url_bypasses_bare_model_names`, and `remote_openai_base_url_still_requires_provider_model_or_tag_syntax` in `rusty-claude-cli`'s `alias_resolution_tests` module; `is_local_openai_compatible_base_url_is_reachable_via_crate_root` in `api`'s `client.rs` confirming the re-export. diff --git a/docs/local-openai-compatible-providers.md b/docs/local-openai-compatible-providers.md index a109cc80a1..54ea5764e5 100644 --- a/docs/local-openai-compatible-providers.md +++ b/docs/local-openai-compatible-providers.md @@ -2,7 +2,7 @@ This guide covers two common offline/local workflows: -1. running Claw against an OpenAI-compatible local model server such as Ollama, llama.cpp, or vLLM; and +1. running Claw against an OpenAI-compatible local model server such as Ollama, LM Studio, llama.cpp, or vLLM; and 2. installing local skills from disk so Claw can discover them without network access. ## Claw is not Claude-only @@ -25,7 +25,8 @@ Routing notes: - Use the `openai/` prefix for OpenAI-compatible gateways when you need prefix routing to win over ambient Anthropic credentials, for example `--model "openai/gpt-4.1-mini"` with OpenRouter. - For local servers, prefer the exact model ID reported by the server (`qwen3:latest`, `llama3.2`, etc.). If your local gateway exposes slash-containing IDs, prefix the exact slug with `local/` so Claw routes through OpenAI-compatible transport while sending the rest verbatim, for example `--model "local/Qwen/Qwen2.5-Coder-7B-Instruct"`. -- If you have multiple provider keys in your environment, `OPENAI_BASE_URL` plus local-looking tags such as `llama3.2` or `qwen2.5-coder:7b` selects the local OpenAI-compatible route; use `local/` for slash-containing local IDs. +- When `OPENAI_BASE_URL` points at a loopback or private-network address (`localhost`, `127.0.0.1`, `10.x`, `192.168.x`, `172.16-31.x`), Claw accepts **any** non-empty model name — bare aliases like `mistral` work with no dot, colon, or slash required. This covers LM Studio and llama.cpp server setups that register models under a plain `--alias`, not just Ollama-style tagged names (`llama3.2`, `qwen2.5-coder:7b`). `local/` is still needed only to preserve slash-containing IDs verbatim (e.g. Hugging Face repo IDs for vLLM/mlx-lm). +- Against a remote/hosted OpenAI-compatible gateway (not local/private), Claw keeps the stricter `provider/model` or dot/colon-tag requirement so a mistyped bare name doesn't silently route to a paid hosted API. - Tool workflows need model/server support for OpenAI-compatible tool calls. Plain prompt smoke tests can pass even when slash/tool workflows still fail because the server returns an incompatible tool-call shape. ## Raw `/v1/chat/completions` smoke test @@ -81,6 +82,17 @@ export OPENAI_API_KEY="local-dev-token" claw --model "qwen2.5-coder" prompt "Reply exactly HELLO_WORLD_123" ``` +## LM Studio + +Start LM Studio's local server (Developer tab → Start Server, default port `1234`), or headless via `lms server start`. LM Studio commonly serves models under a bare alias with no dot or colon (for example `mistral-7b-instruct` or a custom name set in the model settings). + +```bash +export OPENAI_BASE_URL="http://127.0.0.1:1234/v1" +claw --model "mistral-7b-instruct" prompt "Reply exactly HELLO_WORLD_123" +``` + +`OPENAI_API_KEY` can stay unset for LM Studio's default authless local server. Use the exact model identifier LM Studio reports (check `curl $OPENAI_BASE_URL/models`) if the bare alias above doesn't match what you loaded. + ## vLLM or another OpenAI-compatible server Start vLLM with an OpenAI-compatible API server: diff --git a/rust/crates/api/src/client.rs b/rust/crates/api/src/client.rs index 240559e809..538c7fbdd8 100644 --- a/rust/crates/api/src/client.rs +++ b/rust/crates/api/src/client.rs @@ -262,4 +262,21 @@ mod tests { other => panic!("Expected ProviderClient::OpenAi for local model, got: {other:?}"), } } + + #[test] + fn is_local_openai_compatible_base_url_is_reachable_via_crate_root() { + // Regression: this helper must stay `pub` and re-exported from the + // crate root so `rusty-claude-cli`'s model-syntax validation can + // reuse the same local/private-address detection used here to + // waive the API-key requirement. + assert!(crate::is_local_openai_compatible_base_url( + "http://127.0.0.1:1234/v1" + )); + assert!(crate::is_local_openai_compatible_base_url( + "http://192.168.1.50:8000/v1" + )); + assert!(!crate::is_local_openai_compatible_base_url( + "https://openrouter.ai/api/v1" + )); + } } diff --git a/rust/crates/api/src/lib.rs b/rust/crates/api/src/lib.rs index e96e92f830..9fdfbce978 100644 --- a/rust/crates/api/src/lib.rs +++ b/rust/crates/api/src/lib.rs @@ -22,9 +22,9 @@ pub use prompt_cache::{ pub use providers::anthropic::{AnthropicClient, AnthropicClient as ApiClient, AuthSource}; pub use providers::openai_compat::{ build_chat_completion_request, check_request_body_size, estimate_request_body_size, - flatten_tool_result_content, is_reasoning_model, model_rejects_is_error_field, - model_requires_reasoning_content_in_history, translate_message, OpenAiCompatClient, - OpenAiCompatConfig, + flatten_tool_result_content, is_local_openai_compatible_base_url, is_reasoning_model, + model_rejects_is_error_field, model_requires_reasoning_content_in_history, translate_message, + OpenAiCompatClient, OpenAiCompatConfig, }; pub use providers::{ detect_provider_kind, max_tokens_for_model, max_tokens_for_model_with_override, diff --git a/rust/crates/api/src/providers/openai_compat.rs b/rust/crates/api/src/providers/openai_compat.rs index 8fb3969913..e1cca048e5 100644 --- a/rust/crates/api/src/providers/openai_compat.rs +++ b/rust/crates/api/src/providers/openai_compat.rs @@ -1027,7 +1027,12 @@ fn url_host(url: &str) -> &str { host_port.split(':').next().unwrap_or("") } -fn is_local_openai_compatible_base_url(url: &str) -> bool { +/// Returns true when `url`'s host is loopback or a private-network address +/// (`localhost`, `127.0.0.1`, `10.x`, `192.168.x`, `172.16-31.x`). Used to +/// waive auth requirements for local OpenAI-compatible servers (Ollama, LM +/// Studio, llama.cpp, vLLM) that don't expect a real API key. +#[must_use] +pub fn is_local_openai_compatible_base_url(url: &str) -> bool { let host = url_host(url.trim()); if host.eq_ignore_ascii_case("localhost") || host == "::1" { return true; diff --git a/rust/crates/rusty-claude-cli/src/main.rs b/rust/crates/rusty-claude-cli/src/main.rs index 665ce632cf..532f4b29c1 100644 --- a/rust/crates/rusty-claude-cli/src/main.rs +++ b/rust/crates/rusty-claude-cli/src/main.rs @@ -2986,7 +2986,18 @@ fn is_local_openai_model_syntax(model: &str) -> bool { if let Some(rest) = model.strip_prefix("local/") { return !rest.is_empty() && rest.split('/').all(|segment| !segment.is_empty()); } - std::env::var_os("OPENAI_BASE_URL").is_some() && (model.contains(':') || model.contains('.')) + let Ok(base_url) = std::env::var("OPENAI_BASE_URL") else { + return false; + }; + // A local/private OPENAI_BASE_URL (Ollama, LM Studio, llama.cpp, vLLM, ...) + // is trusted to accept any non-empty model name the server actually + // serves, matching the OLLAMA_HOST bypass above. Remote gateways keep the + // stricter dot/colon-tag heuristic so a bare typo doesn't silently route + // to a paid hosted API. + if api::is_local_openai_compatible_base_url(&base_url) { + return !model.is_empty(); + } + model.contains(':') || model.contains('.') } fn config_alias_for_current_dir(alias: &str) -> Option { @@ -19828,4 +19839,37 @@ mod alias_resolution_tests { // Empty model still rejected assert!(validate_model_syntax("").is_err()); } + + #[test] + fn local_openai_base_url_bypasses_bare_model_names() { + let _guard = ollama_env_lock(); + let _ollama_env = EnvVarGuard::unset("OLLAMA_HOST"); + // LM Studio's default port; a bare alias with no dot/colon/slash, + // as LM Studio and llama.cpp commonly register models under. + let _openai_env = EnvVarGuard::set("OPENAI_BASE_URL", "http://127.0.0.1:1234/v1"); + assert!(validate_model_syntax("mistral").is_ok()); + assert!(validate_model_syntax("local-7b").is_ok()); + // Empty model is still rejected even when the base URL is local. + assert!(validate_model_syntax("").is_err()); + } + + #[test] + fn private_network_openai_base_url_bypasses_bare_model_names() { + let _guard = ollama_env_lock(); + let _ollama_env = EnvVarGuard::unset("OLLAMA_HOST"); + let _openai_env = EnvVarGuard::set("OPENAI_BASE_URL", "http://192.168.1.50:8000/v1"); + assert!(validate_model_syntax("mistral").is_ok()); + } + + #[test] + fn remote_openai_base_url_still_requires_provider_model_or_tag_syntax() { + let _guard = ollama_env_lock(); + let _ollama_env = EnvVarGuard::unset("OLLAMA_HOST"); + let _openai_env = EnvVarGuard::set("OPENAI_BASE_URL", "https://openrouter.ai/api/v1"); + // A bare name with no dot/colon/slash against a remote gateway must + // keep failing so a typo doesn't silently route to a paid hosted API. + assert!(validate_model_syntax("mistral").is_err()); + // The existing dot/colon-tag heuristic still applies for remote hosts. + assert!(validate_model_syntax("qwen2.5-coder:7b").is_ok()); + } }