Loading ToolBench
MCP control plane server with health checks, knowledge base search, and audit query capabilities
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Tool names follow verb_noun convention (health.check, kb.search, audit.query) but descriptions are minimal (34-60 chars, below the 50-200 char LLM-optimized baseline). Parameters lack descriptions in schema definitions, only visible in function signatures. No error handling guidance, no output schema documentation, and no tool annotations (readOnlyHint, etc.). The server implements prompts and resources but tool definitions themselves are underdeveloped for production agent use.
Query audit logs with a search string and limit
Health check endpoint that verifies server status and dependencies
Search the knowledge base for documents matching a query
Parameter descriptions missing from schema definitions. 'query', 'top_k', 'q', 'limit' have no descriptions in the JSON Schema, only visible in function signatures. LLMs cannot read Python function signatures; they parse JSON Schema. This violates the requirement that every parameter must have a description.
No output schema documentation. Tools return Pydantic models (HealthCheckResponse, KBSearchResponse, AuditQueryResponse) but the MCP server does not expose their schemas to clients. LLMs cannot plan downstream calls or extract fields without knowing the response structure.
Tool descriptions are too brief (34-60 chars vs. 50-200 char baseline). 'Health check endpoint that verifies server status and dependencies' (60 chars) lacks WHEN to use it and WHAT it returns. 'Search the knowledge base for documents matching a query' (56 chars) omits result format and pagination info.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | <=2025-11-25 | v2 |
No error handling guidance. Tools return model_dump() but no error responses are documented. If kb.search fails (empty KB, adapter error), what does the LLM see? No recovery hints like 'Try a broader query' or 'Check if KB is initialized'.
No tool annotations. Tools are marked READ_ONLY in the metadata but the MCP server does not register readOnlyHint, destructiveHint, or idempotentHint in tool definitions. Agents cannot infer safety properties from the schema alone.