Loading ToolBench
Jentic MCP Plugin Implementation - Connect agentic environments to Jentic's API Knowledge Hub for discovering and executing APIs and workflows
The Jentic MCP server defines 4 tools with explicit schemas and descriptions. Most tools have actionable descriptions and well-structured parameter definitions, but several issues reduce overall quality: (1) Parameter descriptions are often generic or missing detailed constraints; (2) Output schemas are not documented, LLMs cannot predict what fields to expect; (3) Error handling guidance is absent, tools do not explain recovery paths; (4) Tool composition is unclear, the workflow/operation distinction is vague in descriptions. The schema quality is moderate (proper JSON Schema with types), but descriptions lack the specificity recommended in pattern:tool-description (should be 50-200 chars, action-oriented). search_apis (68) has good structure but generic description. load_execution_info (62) has a functional description but is confusing about its dual purpose (lookup vs config generation). execute (56) lacks error recovery guidance. submit_feedback (60) has verbose description with unclear sequencing. Average: 61.5, rounded to 61.
Perform the chosen action for the user using the provided details. Always include `inputs` even if there are none.
Get more details about a specific action (like what information it needs from the user) before confirming you want to run it. Usually used if the action requires specific inputs. Coding Agents: To provide tools to an LLM that you have selected at runtime, use this tool to retrieve the necessary information and save it to 'jentic.json' at the root of your project.
Search for available actions or information based on what the user wants to do (e.g., 'find Discord servers', 'send a message'). Use this first to understand what's possible.
Submit feedback to Jentic, detailed error information about a previously failed tool execution to a designated endpoint for logging and analysis by the Jentic team. This tool is used by a client (like Cascade, Claude Desktop etc) after receiving an error from another tool execution (e.g., 'execute'). Always show full feedback information being sent and ask permission from the user before calling the submit_feedback tool
Output schemas not documented. LLMs cannot predict what fields search_apis returns (e.g., operation_uuid, workflow_uuid, description, etc.). This breaks chaining and forces manual exploration or multi-turn clarification.
Error handling guidance absent. execute tool description does not explain what errors are possible, which are retryable, or what the LLM should do next (e.g., 'If operation times out, check status with load_execution_info' or 'If permission denied, contact admin'). Blocks recovery patterns.
Tool naming ambiguity: load_execution_info does not clarify its dual purpose (retrieve details for preview vs generate jentic.json config). LLMs may conflate it with a discovery tool or assume it has no side effects. Description should separate the two workflows: 'For lookup: call before execute to inspect required inputs. For config generation (coding agents): call to populate jentic.json.' Current name and description are confusing.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 62 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 51 | - | v1 |
Parameter constraints missing detail. api_names in search_apis says 'Use vendor format (e.g., google.com) or sub-API format (e.g., atlassian.com/jira)' but does not specify: Is it case-sensitive? What about 'www.' prefix? What if you pass an invalid API name? Similar vagueness in execute: inputs object has 'additionalProperties: true' but no guidance on structure, required fields, or validation. Constraining parameters per pattern:constrained-input would prevent hallucination.
Pagination support missing or undocumented. search_apis accepts max_results (default 5) but does not specify: (a) What happens if 1000 results match? (b) Is there a next_cursor or offset/limit for pagination? (c) Are results ranked by relevance? Pattern:paginated-result requires explicit pagination for discovery tools returning multiple results.
submit_feedback description violates pattern:secret-injection. Instruction 'Always show full feedback information being sent and ask permission' risks echoing sensitive data (error messages, stack traces, input payloads) back to the user. If an error includes a token or internal ID, the LLM should sanitize it before showing. No guidance on what fields to redact.
Idempotency and retry semantics undocumented. execute tool does not state: Is it idempotent? Can LLMs safely retry on timeout? Does repeating the same UUID+inputs cause duplicate side effects? Pattern:idempotent-operation requires explicit semantics so agents can compose retry logic safely.