Loading ToolBench
Flexible SAP OData MCP Server - Intelligent, dynamic interaction with any SAP OData service without restrictions
This SAP OData server has significant definition gaps that prevent confident production use. While all 10 tools have basic names and descriptions, critical issues emerge across naming clarity, parameter documentation, output schemas, and error handling. Tool names like 'discover_entity_sets' and 'analyze_entity_structure' start with verbs but lack the specificity needed for multi-step agent reasoning. Most parameters are described (positive), but several critical issues reduce confidence: (1) No input schemas visible in source code for validation, parameter definitions exist in the rubric but are not explicitly shown in mcp_server.py tool registration; (2) Descriptions are present but generic (averaging ~80 chars, well below the 194-char production baseline) and lack actionable context; (3) No output schemas documented, agents cannot predict what fields to expect from tool responses; (4) Error handling is minimal, exceptions are caught but recovery guidance is absent; (5) No permission gates on destructive operations (delete_entity accepts any input); (6) STDIO transport is a hard blocker for remote accessibility. The server reads well-structured and implements legitimate OData logic, but tool definitions are not production-ready by Arcade standards.
Analyze entity structure by sampling data
Create a new entity in the specified entity set
Delete an entity from the specified entity set
Discover all available OData services on the SAP system
Dynamically discover all available entity sets for a service
Intelligently find which service contains a specific entity
Get and parse OData metadata
No input schemas visible in source code. While the rubric lists parameter definitions (e.g., read_entity has 'service', 'filter', 'top', 'skip', 'select', 'orderby'), the actual tool registration in mcp_server.py does not show explicit JSON Schema definitions with types and constraints. Schemas appear to be inferred from function signatures rather than formally declared.
Output schemas not documented. Tool descriptions state WHAT they do (e.g., 'Read data from an entity set') but do NOT specify what fields are returned or what structure the LLM should expect. For example, read_entity returns JSON from the OData service, but the tool definition does not document whether results are wrapped in a 'value' array, what pagination tokens are included, or what metadata fields are present. This forces LLMs to guess and makes downstream tool composition unreliable.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 39 | 2026-07-28+ | v2 |
Get and cache service document
Read data from an entity set with filtering and pagination
Update an existing entity
Destructive operation (delete_entity) has no permission gate, confirmation step, or dry-run mode. The tool accepts 'entity_set', 'entity_key', and optional 'service' but provides no mechanism to prevent accidental deletion. An agent could invoke delete_entity without user confirmation, leading to irreversible data loss.
Error handling provides minimal recovery guidance. The source code catches exceptions and raises generic messages like 'HTTP 404: ...' or 'Request failed: ...', but does not tell the agent what to do next. For example, if discover_all_services fails, the error does not suggest 'Try discover_entity_sets() on a known service instead' or 'Check authentication credentials.' This violates the recovery-guide pattern.
Parameter descriptions are generic and lack actionable detail. For example, 'filter' on read_entity is described as 'Optional OData filter expression', but does not explain the syntax, provide an example, or document valid operators. LLMs cannot reliably generate OData filter syntax from such sparse hints. Descriptions should follow the pattern: 'OData filter expression using operators like eq, ne, gt, lt (e.g., "status eq 'open' and priority gt 3")'.
Tool descriptions are too brief (averaging ~40-50 chars, well below the 194-char production baseline). For example, 'Analyze entity structure by sampling data' does not explain WHY an agent would call this tool, WHEN to call it relative to other discovery tools, or WHAT the output structure looks like. Descriptions should be expanded to ~100-150 chars to provide sufficient context for LLM tool selection.
No pagination support documented. read_entity accepts 'top' and 'skip' parameters but does not declare max limits, describe the pagination semantics, or document how to detect end-of-results. If an agent requests 10,000 records, there is no guidance on safe page size or whether the tool enforces a ceiling. Production tools should document: 'top accepts 1 - 1000 (default 20); skip accepts 0 or positive integer; if returned count < top, you have reached the end.'
create_entity and update_entity require 'data' as a free-form object parameter. The description 'Entity data to create' does not specify what fields are valid, which are required, or what types they should be. An agent cannot reliably construct this object without analyzing the entity structure first. A more actionable description: 'Object with entity field names as keys; required fields depend on the entity set (call analyze_entity_structure to discover them).'
No audit logging or permission checking visible in source. write and destructive operations (create_entity, update_entity, delete_entity) do not log who called them, with what data, or when. For compliance and incident response, these operations should emit structured audit events. The source shows no audit trail mechanism.
Credentials are injected via environment variables (username, password in __init__), which is correct. However, if these are ever logged or returned in tool responses, they risk leaking into traces. The source does not show explicit guards against returning authentication tokens. This risk should be mitigated with a comment or validation.
Naming could be clearer for discovery tools. 'discover_entity_sets' is clear, but 'analyze_entity_structure' is vague, does it modify the entity? Does it return schema or sample data? A more specific name like 'get_entity_structure' or 'sample_entity_fields' would help LLMs understand intent without reading the description.