Loading ToolBench
MCP server for I14Y — Switzerland's national open-data metadata catalogue (DCAT-AP-CH)
Strong foundation with complete schemas, clear descriptions, and proper parameter constraints across all 12 tools. All tools follow verb_noun naming (search_, get_, list_). Descriptions average ~120 chars, within the 10-1024 baseline. All parameters have types and descriptions. Pagination is well-implemented (page, page_size with min/max bounds). Language enum is consistent across tools. Main gaps: output schemas are not explicitly documented in the code; error handling lacks recovery guidance; no tool annotations (readOnlyHint present but not visible in schema registration); descriptions could be more LLM-optimized with dependency hints.
Check the status of the I14Y API and the last successful request timestamp.
Fetch a single concept record with its code list entries.
Fetch a single data service record with its endpoint URLs and metadata.
Fetch a single dataset record with all its distributions and metadata.
Fetch the distributions (download URLs, formats, licences) of a dataset.
List catalogues in the I14Y system, paginated.
List concepts (controlled vocabularies, code lists) in the catalogue, paginated.
Output schemas not explicitly documented in code. Tool descriptions state what is returned (e.g., 'Fetch a single dataset record with all its distributions and metadata') but the actual response structure is not visible in the provided source. LLMs cannot plan downstream tool calls without knowing which fields to expect.
Error handling lacks recovery guidance. No evidence of try-catch blocks, error categorization (retryable vs. fatal), or actionable error messages in the visible code. When a dataset_id is invalid or not found, the LLM receives no hint about what to do next (e.g., 'Try search_catalog() first').
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | <=2025-11-25 | v2 |
List data services (APIs, interfaces) in the catalogue, paginated.
List datasets in the catalogue, paginated.
List public services in the catalogue, paginated.
List publishers (organizations) in the catalogue, paginated.
Search Switzerland's national metadata catalogue for data resources. The primary entry point: use this to find out *who* publishes data on a topic before looking for a specific dataset.
Descriptions lack dependency hints and LLM-optimized guidance. E.g., search_catalog says 'use this to find out *who* publishes data' but does not explicitly state 'Call this first before get_dataset() if you only have a topic, not a dataset ID.' This forces agents to discover the call sequence through trial and error.
Tool annotations (readOnlyHint) declared in feature flags but not visible in schema registration code. All tools are read-only (Risk: READ_ONLY noted), but the MCP schema registration does not show explicit annotation metadata that clients can parse to optimize caching or safety policies.
api_status tool has empty input schema ({}). While this is valid for a no-argument tool, the description is minimal ('Check the status of the I14Y API and the last successful request timestamp'). No guidance on what the response contains or when to call it in a workflow.