Loading ToolBench
Multi-server MCP implementation for ATLAS brain system with B2B churn prediction, memory management, and invoicing capabilities
ATLAS MCP server has severe definition quality gaps. Tool descriptions are present but generic (avg ~60 chars, below 194-char baseline). Input schemas visible but lack parameter-level type definitions and descriptions, most params have only type and description at top level, missing granular constraints. No output schemas documented. Tools exhibit composition issues: execute_intent, execute_action, execute_tool, and execute_tool_by_intent overlap significantly, creating ambiguity for LLM selection. Error handling absent, no recovery guidance, categorization, or actionable error messages. Security concerns: no evidence of permission gates, audit trails, or secret injection patterns. Naming is verb-forward but lacks clarity on mutual exclusivity and parameter type suffixes.
Book an appointment with accumulated context from conversation
Execute a direct action on a capability
Execute a parsed intent via ActionDispatcher
Execute a built-in tool
Execute a tool based on intent target_name. Maps intent target names to tool registry names and executes
Parse intent from natural language query
Fast intent routing using semantic embeddings. Classifies queries into: device_command, tool_use, or conversation
Four tools (execute_intent, execute_action, execute_tool, execute_tool_by_intent) have overlapping responsibilities. LLMs cannot disambiguate when to call which. Violates single-responsibility principle.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. Violates response-shaper pattern.
Parameter descriptions are minimal (avg 40-60 chars vs 72-char baseline). 'intent' param in execute_intent lacks format/structure details. 'params' in execute_action and execute_tool are free-form objects with no schema or constraints.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | <=2025-11-25 | v2 |
No error handling guidance. Tools do not indicate retryability, user-fixable errors, or recovery steps. Agents cannot self-correct on failure.
No evidence of permission gates, audit trails, or secret injection. book_appointment accepts customer_email and customer_phone as parameters, sensitive data should not appear in tool params.