Loading ToolBench
Open Japanese tourism dataset + MCP server — 13,394 tourist spots × 18 languages, ~20,000 accommodations, 690 officially-designated cultural records. All 47 prefectures, all from public sources. Free, no API key.
Critical failures across all dimensions. Tool definitions are inferred from test fixtures only, no explicit registration visible in source code. All 19 tools lack documented input schemas (cannot verify parameter types, constraints, or descriptions). Tool descriptions are generic placeholders (10-40 chars) with no actionable guidance. No output schemas documented. No error handling patterns. No parameter validation rules. This server cannot be evaluated against production quality baselines because the tool definitions themselves are not visible in the codebase, only test mocks exist.
Get detailed description of a tourist attraction
Get Destination Management Organization information
Get multiple entities in bulk
Get full entity information
Get events information
Get festival information
Get hotel and accommodation information
Tool definitions not visible in source code, only test fixtures exist. Cannot verify actual tool registration, schemas, or parameter definitions. Per HARD SCORING RULE: if tool definitions are inferred rather than directly visible, cap per-tool scores at 50. Since no actual definitions are found, score is 0 for schema dimension.
All tool descriptions are generic placeholders (10-40 characters). Examples: 'Get detailed description of a tourist attraction' (48 chars, but lacks WHEN to use, WHAT it returns, or dependencies). No descriptions explain what data structure is returned, what parameters are required, or how to chain with other tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 35 | 2026-07-28+ | v2 |
Get Japan Heritage site information
Get local food and cuisine information
Get local specialty product information
Get multilingual information for attractions
Get tourist spots information
Get traditional arts and cultural information
Get transportation information
Check feasibility of a travel plan
Resolve an entity to its full information
Search for attractions in a specific area
Hybrid search combining keyword and semantic search
Semantic search for attractions using embeddings
No input parameter schemas visible. Only get_description shows a single parameter 'qid' with type string and minimal description. No other tools' parameter definitions are documented. Cannot verify: enums, constraints, required vs optional, min/max bounds, format patterns, or parameter descriptions.
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields without knowing response structure. No evidence of pagination support (limit, offset, total_count, next_cursor) despite 19 tools that likely return lists.
No error handling patterns visible. No recovery guidance, error classification (retryable vs fatal), or actionable error messages. No dry-run or confirmation step for potentially destructive operations. Per pattern:recovery-guide, errors must tell LLM what to do next.