Loading ToolBench
SAP Gateway MCP Server for OData service integration and AI-powered data access
The SAP MCP Server defines 4 tools with schemas present, but exhibits significant gaps in parameter descriptions, output documentation, and error recovery guidance. Tool naming follows verb_noun convention correctly (sap_authenticate, sap_get_entity, sap_query, sap_list_services). Schemas are visible and properly typed for most parameters. However, parameter descriptions are minimal or missing context about constraints, formats, and error recovery. Output schemas are entirely undocumented, callers have no formal specification of what fields to expect. Error handling is minimal; execute() methods catch exceptions generically without actionable recovery guidance. The server appears functional but falls short of production-grade tool quality expected for enterprise SAP integration.
Authenticate with SAP Gateway using username and password
Retrieve a single entity from SAP OData service by key (e.g., OrderID)
List all available SAP OData services configured in services.yaml
Query SAP OData service entity sets with optional filters
sap_authenticate has empty input schema (no properties, no required fields) with no documentation explaining that credentials come from environment/config, not parameters. This violates secret-injection pattern and leaves LLM confused about how to invoke it.
No output schemas documented for any tool. Callers cannot see what fields to expect (e.g., does sap_query return 'results' or 'data'? is there a 'total_count' or 'next_cursor' for pagination?). This forces LLMs to guess at response structure.
sap_query 'select' and 'filter' parameters lack format documentation. What is the exact OData syntax? Example: 'Name eq \'John\'' or 'Name=John'? LLMs will hallucinate syntax.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 45 | - | v1 |
sap_query 'top' and 'skip' parameters lack min/max constraints. Can 'top' be 0? 10000? Unbounded numbers let LLMs pass values that break the SAP API.
Error handling is generic. Code in auth_tool.py and entity_tool.py catches Exception and returns {'success': False, 'error': str(e)}. Stack traces and low-level errors offer no recovery guidance to LLMs ('User not found. Try search_users() first.' is the correct pattern).
sap_get_entity and sap_query do not validate that service and entity_set exist before making API calls. While error responses mention 'Service not found', they use generic exception catching elsewhere. Validation should be explicit with clear, actionable errors.
No pagination guidance in tool descriptions. sap_query accepts 'skip' and 'top' but does not state that it caps results at a default limit or explain when to use pagination. Large queries may blow context windows without explicit warnings.