Loading ToolBench
GLM-first, free-tier-aware multi-provider AI cluster for Codex, with an optional bounded AI Team execution plane.
Server has 17 tools with basic schemas and descriptions, but significant quality gaps. Most tool descriptions are present (194-char baseline met for ~60% of tools) but lack actionable context for LLM selection. Parameter descriptions exist but are often generic ('List of...', 'Array of...'). No output schemas documented. Error handling guidance absent. Tool naming is verb-first (good), but several tools combine multiple concerns (e.g., record_provider_failure + record_provider_success should be unified). No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Security: API keys are environment-injected (correct), but no permission gates or audit trail declarations visible. Composition: tools are reasonably single-purpose, but chaining IDs are not consistently returned (e.g., run_project_task returns compact result but unclear if model/provider IDs are included for downstream routing).
Discovers available models from a provider
Evaluates quality policy compliance of a worker result
Retrieves current fabric state and configuration
Retrieves mission control status and active missions
Retrieves health status and cooldown information for a provider
Retrieves usage summary for a time period
Plans an AI team execution with multiple workers and coordination strategy
Preview a project task without executing it, showing estimated tokens and routing decision
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., does run_project_task return model_id, provider, tokens_used?). Violates pattern:tool and pattern:response-shaper.
Parameter descriptions are generic ('List of...', 'Array of...', 'Object with...') and lack actionable constraints. No enums for constrained fields (e.g., budget enum values 'low|normal|deep' are documented but not enforced in schema). Violates pattern:constrained-input and pattern:tool-description.
No error handling guidance. Tools lack recovery hints (e.g., if provider fails, which tool to call next?). No categorization of errors as retryable vs user-fixable. Violates pattern:recovery-guide and pattern:error-classification.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Records a provider failure for health tracking and cooldown management
Records a successful provider call for health tracking
Records a usage event in the ledger for tracking and billing
Routes a model request to the best available provider based on budget mode, requirements, and health status
Runs diagnostic checks on server health and configuration
Executes a project task with specified model, messages, and configuration
Executes a routine maintenance workpack
Updates fabric state with new configuration
Performs web search using X.AI Grok search capability
Tool annotations missing. No readOnlyHint, destructiveHint, or idempotentHint declared. LLMs cannot infer which tools are safe to retry or which modify state. Violates current MCP spec (2026-07-28) tool annotation requirement.
Composition: record_provider_failure and record_provider_success are separate tools doing inverse operations. Should be unified into record_provider_event with a status enum. Violates pattern:tool (single responsibility).