AI application framework with desktop (Electron) and web app support, featuring extensible architecture for AI models, conversations, and downloads
Ax-Studio exposes 8 tools for proxy server and provider management via STDIO. Tool names follow verb_noun convention (start_server, stop_server, get_server_status, register_provider_config, etc.), which is positive. However, critical gaps undermine quality: (1) Descriptions are present but generic and lack actionable context, e.g., 'Start the proxy server with specified configuration' does not explain WHEN to call it, what prerequisites exist, or what happens on failure. (2) Input schemas are visible but incomplete: parameters lack type constraints (e.g., 'trusted_hosts' is array but no item type; 'models' is array with no schema). (3) No output schemas documented, LLMs cannot predict what these tools return. (4) No error handling guidance, tools do not indicate retryability, user-fixable vs. fatal errors, or recovery steps. (5) Security concerns: 'api_key' and 'api_key' parameters expose secrets as tool inputs, violating secret-injection pattern. (6) No pagination, no idempotency hints, no tool annotations (readOnlyHint/destructiveHint). Per-tool analysis: start_server and register_provider_config have the most complex schemas but still lack nested type definitions; stop_server and get_server_status are trivial (empty input). Overall, this is a functional but immature tool set that would require significant hardening before production use.
Abort a remote stream by stream identifier
Get the current status of the proxy server (running or not)
List all registered provider configurations
Register a provider configuration with API key, base URL, custom headers, and models
Register multiple provider configurations in batch
Start the proxy server with specified configuration (host, port, prefix, API key, trusted hosts, CORS settings, verbose logging, proxy timeout)
Stop the proxy server
API keys exposed as tool parameters (start_server, register_provider_config, register_provider_configs_batch). Secrets in parameters are logged in traces and prompt history, violating secret-injection pattern.
No output schemas documented for any tool. LLMs cannot predict return types, forcing them to guess at response structure and plan downstream calls blindly.
Descriptions are generic and lack actionable context. E.g., 'Start the proxy server with specified configuration' does not explain prerequisites, failure modes, or when to use this vs. alternatives. Average description length ~60 chars; baseline is 194 chars.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Unregister a provider configuration by provider name
Input schemas lack nested type definitions and constraints. E.g., 'trusted_hosts' is array but no itemType; 'models' is array with no schema; 'custom_headers' is array with no structure. Parameters cannot be validated by LLMs.
No error handling guidance. Tools do not indicate retryability, user-fixable vs. fatal errors, or recovery steps. LLMs cannot determine next action on failure.