Loading ToolBench
Python SDK for Metorial - The open source integration platform for agentic AI
This is a Python SDK wrapper around the Metorial REST API, NOT an MCP server. The SDK exposes 17 tools derived from REST endpoints via auto-generated code (src/metorial/_generated/magnetar/endpoints/). Tools have names and descriptions, but critical quality issues undermine production readiness: (1) Parameter descriptions are uniformly generic ('optional') with no detail on constraints, formats, enums, or semantics; (2) No visible input schema validation, parameters like 'limit', 'order', and 'created_at' appear as type hints only, without JSON Schema or constraint documentation; (3) Output schemas are not documented, no indication of response structure, fields, or whether pagination is supported; (4) Many parameters are vague (e.g., 'created_at' as Dict[str, Any], unclear if it's a filter, range, or timestamp); (5) No error recovery guidance, descriptions do not explain what failures look like or how to retry; (6) Composition is poor, 17 tools with overlapping names (e.g., 'list_dashboard_instance_tool_calls' vs 'list_management_instance_tool_calls' vs 'list_tool_calls') require significant LLM reasoning to distinguish; (7) Tool naming is inconsistent and hierarchical rather than flat verb_noun pattern; (8) Security: no indication of how authentication is handled, whether rate limits exist, or how secrets are injected. The SDK is auto-generated from an API spec, which explains the uniformity but does not excuse the low quality. For an MCP server, this would be unacceptable.
Create tool call. Creates a new tool call in a session by invoking a specific tool.
Create tool call. Creates a new tool call in a session by invoking a specific tool.
Create tool call. Creates a new tool call in a session by invoking a specific tool.
Get consumer tool call. Retrieves one tool call belonging to an identity owned by the authenticated profile actor.
Get provider tool. Retrieves a specific provider tool by ID.
Get tool call. Retrieves a specific tool call by ID.
Get provider tool. Retrieves a specific provider tool by ID.
Parameter descriptions are uniformly trivial ('optional') with no semantic detail on constraints, formats, ranges, or valid values. Example: 'created_at' as Dict[str, Any] with description 'optional' tells LLMs nothing about whether it's a filter, range query, or timestamp.
Output schemas are completely undocumented. No indication of response structure, field types, pagination support, or what data structure an LLM should expect. This forces agents to blindly invoke tools and parse responses via trial-and-error.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 34 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Get tool call. Retrieves a specific tool call by ID.
Get provider tool. Retrieves a specific provider tool by ID.
Get tool call. Retrieves a specific tool call by ID.
List consumer tool calls. Returns read-only tool-call activity for identities owned by the authenticated profile actor.
List provider tools. Returns a paginated list of provider tools. By default returns tools from the latest version. Use optional filters to get tools for a specific version.
List all tool calls. Returns a paginated list of tool calls across all sessions.
List provider tools. Returns a paginated list of provider tools. By default returns tools from the latest version. Use optional filters to get tools for a specific version.
List all tool calls. Returns a paginated list of tool calls across all sessions.
List provider tools. Returns a paginated list of provider tools. By default returns tools from the latest version. Use optional filters to get tools for a specific version.
List all tool calls. Returns a paginated list of tool calls across all sessions.
Tool naming hierarchy is confusing and inconsistent. 17 tools with prefixes like 'list_dashboard_instance_', 'list_management_instance_', 'list_' create significant cognitive load for LLMs distinguishing between nearly identical names. No clear verb-noun pattern, names are deeply hierarchical (prefix_prefix_prefix_action) rather than flat.
Many parameters accept vague, overloaded types. Example: 'order' as Optional[str], 'created_at' as Optional[Dict[str, Any]], 'provider_ids' as Optional[Union[str, List[str]]], no enum values specified, no format constraints documented. LLMs will hallucinate invalid values.
No error handling guidance. Descriptions do not explain when calls fail (e.g., invalid ID, permission denied, rate limited), what recovery looks like, or whether retries are safe. Critical for agent reliability.
Pagination not documented. 'list_*' tools accept 'limit', 'after', 'before', 'cursor' but descriptions do not clarify pagination semantics (cursor vs offset, whether total count is returned, max page size). LLMs cannot plan multi-page queries safely.
No indication of required vs optional dependencies. 'instance_id' and 'provider_version_id' appear in many tools but parameter descriptions do not clarify their relationship or when each is mandatory.
No visible security declarations. No indication of required permissions, scope declarations, or how authentication tokens are managed. If exposed via MCP, tools could bypass authorization if not gated properly.