Loading ToolBench
Lightweight OpenAI-compatible LLM gateway with routing, fallback chains, and optional web tools
Fleet-gateway exposes 6 tools with reasonable naming (verb-first: chat, search, scrape, health, list_models, list_capabilities) and descriptions present for all. However, schemas are incomplete: chat has full input schema with types and descriptions; search, scrape, health, list_models, list_capabilities lack visible input schemas in the provided code. Parameter descriptions are present for chat but absent for others. Output schemas are not documented. Error handling is minimal, no recovery guidance or actionable error messages. The server is HTTP-based (good for protocol readiness) but tool definitions appear inferred from handler code rather than explicit MCP tool registration, capping individual tool scores at 50 where schemas cannot be verified.
Call an LLM model or capability with messages, routing through fallback chains
Check gateway health status and availability of models and tools
List available capability aliases and their routing chains
List all available models with their capabilities and availability status
Scrape a URL and return its content as Markdown via Firecrawl
Search the web via SearXNG
Input schemas missing for 5 of 6 tools (search, scrape, health, list_models, list_capabilities). Only chat has a visible, complete input schema with types and parameter descriptions. Without schemas, LLMs cannot validate inputs or understand parameter constraints.
Output schemas not documented for any tool. LLMs cannot plan downstream tool calls or extract required fields (e.g., does search return 'channel_id' or 'channel.id'?). Responses appear to be raw HTTP JSON without field documentation.
Error handling lacks recovery guidance. Code shows generic 503 'All backends failed' and 400 'Invalid JSON body' without actionable next steps. No error categorization (retryable vs. user-fixable vs. fatal) to guide agent behavior.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Tool definitions appear inferred from HTTP handler code (_handle_chat, _handle_search) rather than explicitly registered via MCP tool registration. No visible MCP server initialization or tool registration calls in provided code. This suggests tools may not be properly exposed to MCP clients.
Parameter descriptions incomplete. 'search' tool has query, num_results, language, categories but descriptions are minimal (e.g., 'Search query string (required)' lacks format/length constraints). 'scrape' only documents 'url' with no guidance on supported formats or size limits.