Loading ToolBench
MCP server for accessing Israeli government public data via data.gov.il API
Server has 10 tools with basic descriptions and parameter schemas visible in source. However, critical gaps limit quality: (1) Most descriptions are under 100 chars and lack context on WHEN to use each tool or what distinguishes similar tools (e.g., package_search vs package_show). (2) Parameter descriptions are minimal or absent, e.g., 'fq' in package_search has description 'Filter query' but no guidance on syntax or valid values. (3) No output schemas documented, LLMs cannot predict response structure for planning. (4) No error handling guidance, tools call raise_for_status() but provide no recovery hints. (5) fetch_data_gov_il is not async like others, creating inconsistency. (6) No tool annotations (readOnlyHint, etc.) despite all being read-only. Naming is reasonable (verb_noun pattern), but descriptions fall short of the 50-200 char LLM-optimized baseline. Average tool score ~52.
Search a datastore resource.
Fetch data from the Israeli government public API (data.gov.il) based on a dataset name query
Get the list of licenses available for datasets on the site.
Get names of all organizations.
Get details of a specific organization.
Get a list of all package IDs (datasets).
Find packages (datasets) matching query terms.
No output schemas documented. LLMs cannot predict response structure, making downstream tool chaining and data extraction error-prone.
Parameter descriptions are minimal or missing context. E.g., 'fq' (filter query) lacks syntax guidance; 'records_format' in datastore_search has no enum constraint or valid value list.
No error handling guidance. Tools call raise_for_status() but provide no recovery hints (e.g., 'If dataset not found, try search_packages() first'). LLMs receive raw HTTP errors with no actionable next step.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | <=2025-11-25 | v2 |
Get metadata about one specific package (dataset).
Find resources based on their field values.
Get the CKAN version and a list of installed extensions.
Tool descriptions lack context on WHEN to use each tool. Similar tools (package_search vs package_show, organization_list vs organization_show) have identical description patterns, making LLM selection ambiguous.
fetch_data_gov_il is synchronous (def) while all others are async (async def), creating inconsistency in the tool interface and potential runtime issues.