Loading ToolBench
Kolada MCP presents a well-organized server with 9 coherent read-only tools for Swedish municipal statistics. Tool naming is consistent and verb-driven (list_, get_, search_, filter_, fetch_, analyze_, compare_). All tools have descriptions and input schemas are visible in the source. However, several critical gaps lower the score: (1) output schemas are not documented, responses are not specified anywhere in tool definitions, (2) parameter descriptions are present but sparse (averaging ~40 chars vs production baseline of 72), (3) no error handling patterns or recovery guidance visible, (4) no pagination support despite tools returning potentially large result sets, (5) some parameter constraints are vague (e.g., 'cutoff_value' accepts 'number' but no range specified). The server demonstrates solid fundamentals (HTTP transport, proper schema structure, read-only operations) but lacks the production-grade refinements needed for 70+.
Analyze a KPI across multiple municipalities with rankings and statistics.
Compare two KPIs across municipalities. Single year: calculates difference. Multiple years: calculates correlation.
Fetch raw KPI data for specified municipalities.
Filter municipalities based on KPI value threshold.
Get detailed metadata for a specific KPI.
Get all KPIs within a specific operating area.
List all municipalities or filter by type.
Output schemas completely undocumented. Tools lack return type definitions, making it impossible for LLMs to plan downstream calls or extract expected fields. compare_kpis mentions 'calculates correlation' but no structured return is specified.
No pagination support documented or visible for tools returning lists. list_operating_areas, get_kpis_by_operating_area, list_municipalities, filter_municipalities_by_kpi, and analyze_kpi_across_municipalities accept no limit/offset or cursor parameters, risking context window exhaustion.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 72 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 51 | - | v1 |
List all operating areas (KPI categories) with counts. Returns thematic categories of KPIs available in the Kolada database.
Search for KPIs using semantic similarity. Uses embeddings to find KPIs semantically similar to the query.
Parameter constraints vague or missing. 'cutoff_value' in filter_municipalities_by_kpi is 'number' with no range specified. 'limit' in search_kpis has no documented max. 'year' and 'years' parameters lack format or range guidance (e.g., '2020,2021,2022' vs '2020-2022'?). LLMs cannot validate inputs without explicit constraints.
Parameter descriptions are sparse (averaging ~30-50 chars vs production baseline 72 chars). Examples: 'The KPI identifier' (19 chars), 'Natural language search query' (29 chars), 'Comma-separated municipality IDs' (33 chars). Missing context on what constitutes a valid KPI ID, expected search query length, or exact format of comma-separated values.
No error handling or recovery guidance. Tools lack descriptions of what errors can occur (e.g., invalid KPI_id, malformed municipality list, timeout on API call) or actionable next steps. Pattern:recovery-guide missing.
Tool descriptions under-specify dependencies. search_kpis says 'Uses embeddings to find KPIs semantically similar to the query' but does not explain what embeddings model is used, whether offline or requires network access, or potential latency implications. analyze_kpi_across_municipalities mentions 'rankings and statistics' but does not specify sorting order or statistical formulas.