Loading ToolBench
MCP Plugin for SAP CAP (Cloud Application Programming Model) that exposes CAP services, entities, and operations as MCP tools and resources
CAP MCP Plugin demonstrates solid foundational tool definition work with proper schema structures and parameter descriptions. All 6 tools have input schemas with typed parameters and descriptions. Tool names follow verb_noun convention (query, get, create, update, delete, execute). However, several quality gaps reduce the score: (1) Tool descriptions are minimal (12-68 characters, below the 50-200 char LLM-optimized baseline); (2) Parameter descriptions lack operational context and constraints; (3) Output schemas are not documented in the tool definitions; (4) Error handling and recovery guidance are not evident; (5) The generic 'Service_Entity_*' prefix obscures the actual domain (OData-based CAP entity management). The server appears well-engineered for its domain (SAP CAP integration) but under-optimized for LLM interaction patterns.
Create a new entity instance
Delete an entity instance
Retrieve a single entity by its key
Query entity with structured filters, select, orderby, top/skip
Update an existing entity instance
Execute a CAP function or action operation with parameters
Tool descriptions are significantly below LLM-optimized length. Service_Entity_get (30 chars) and Service_Entity_delete (30 chars) fail the 50-200 char baseline. Descriptions lack operational context: when should the LLM choose this tool over alternatives? What constraints apply? What does it return?
Parameter descriptions lack actionable constraints and operational guidance. Example: 'filter' param has description 'OData $filter expression for querying entities' but does not explain valid syntax, examples of valid filters (without putting example values in description per best practice), or what happens when filter is invalid. 'top' param states 'Maximum number of records to return (max 200)' but does not explain pagination strategy, behavior when top > actual results, or whether ordering is guaranteed.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | 2025-01-23+ | v1 |
Output schemas are not documented in tool definitions. LLM cannot infer what fields are returned by Service_Entity_query, Service_Entity_get, or Service_Entity_create. Without documented output structure, agents cannot plan downstream tool calls or extract required data (e.g., which fields are available for filtering, what IDs are returned for chaining).
Error handling and recovery guidance absent. Tools do not declare what errors they can raise, how to interpret them, or what the LLM should do next. Example: if Service_Entity_query receives an invalid OData filter, what error is returned? Is it retryable? Should the LLM try a different filter syntax or ask the user for clarification?
Generic tool name prefix 'Service_Entity_*' obscures domain semantics. An LLM reading these names cannot infer: what is a 'Service'? What is an 'Entity'? Are these generic data operations or CAP-specific? Domain-specific naming (e.g., 'query_cap_entity', 'create_business_object') or documentation linking these to actual CAP data models would improve clarity.
Destructive tools (Service_Entity_delete, potentially Service_Entity_update and Service_Entity_create) lack confirmation or dry-run patterns. There is no confirmation_request step or dry-run capability documented. An LLM could accidentally delete critical data without a chance to verify.
Service_Entity_get parameter 'keys' is described as 'Entity key fields required for lookup' but does not specify which fields constitute a key, what happens if partial keys are provided, or what happens if no key exists. This is ambiguous, the LLM cannot reliably construct valid key objects.
Service_Entity_query does not document pagination behavior. What is returned when results exceed 'top'? Is there a 'next_cursor' or 'total_count' field? Without this, agents cannot reliably iterate through large result sets.
Service_Operation_execute has vague parameter schema: 'parameters' is described as 'Operation-specific parameters derived from CDS definition' but does not specify which operations are available, what parameters each operation expects, or what valid values are. This is opaque to the LLM and likely to result in malformed calls.