Loading ToolBench
A FastMCP server providing multiple travel-related services including product search, image/video search, map generation, and currency exchange rate queries.
This server exposes 9 travel/exchange tools via fastmcp with HTTP+SSE transport. Definitions show moderate quality with documented tool names and basic descriptions in Chinese and English, but critical gaps emerge across naming conventions, parameter descriptions, and output schemas. 8 of 9 tools lack verb-prefixed names (naming convention baseline: 90% of A+ tools start with action verbs). Most parameter descriptions are present but sparse (under 72 chars baseline). Output schemas are undocumented, the code uses type hints in function signatures (e.g., `Dict[str, List[str]]` for image_search) but never explicitly documents the returned structure for LLM consumption. Error handling is minimal: exceptions return empty defaults (e.g., `return {keyword: [] for keyword in keywords}`) with no guidance on retry, user action, or root cause. Tool composition is weak, multiple tools serve overlapping purposes (two product query variants at `/v2/product`, two image/video search tools), and no clear dependency chaining. Parameters accept only strings/lists with minimal validation (e.g., currency codes accept arbitrary strings; no enum constraint visible). Per-tool analysis reveals consistent structural weaknesses rather than isolated defects.
使用产品编号查询旅行产品的详情，返回信息包含景点酒店机票的具体描述
使用目的地查询旅行产品框架，返回信息包不包含景点酒店机票的具体描述
使用目的地查询旅行产品详情，返回信息包含景点酒店机票的具体描述
根据关键字列表搜索对应的图片
根据关键字列表搜索对应的视频
按照途经点生成路书地图
查询多个地点(带日期)的天气信息，如果一个地点涉及多个日期，请将列出每个地点日期
8 of 9 tools lack verb-prefixed names. Names are in Chinese without action verbs (根据...查询, 按照...检索, 使用...查询). Baseline: 90% of A+ tools start with get_, create_, search_, list_, etc. LLMs infer intent from names before reading descriptions.
Output schemas are undocumented. Tools return typed values (Dict, str, List) in function signatures but no documentation of field names, structure, or format for LLM consumption. Baseline: 100% of A+ tools have documented return types.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 43 | - | v1 |
汇率查询，根据货币代码，查询换算金额
获取地点的经纬度信息
Parameter values lack enum constraints. Currency codes, country/province/city strings, and status fields accept arbitrary input. Baseline pattern: 'declare as enum when parameter accepts known set of values.' Current code shows no validation.
Error handling returns silent defaults (empty strings, empty dicts) with no guidance to LLM. Exceptions caught, logged, but responses contain no error message, cause, or retry instruction. Baseline: 'Error responses must tell the LLM what to do next.'
Tools #6 and #7 are nearly identical (both query travel products by location, differ only in detail level). Naming does not distinguish them (both use Chinese 查询 without verb prefix). Baseline: 'When multiple tools operate on same resource, names must make distinction obvious.' This forces LLM to read full descriptions to disambiguate.
Descriptions contain example values (e.g., 'U123456' for product_num, 'Tianjin,CN' for locations, '116.407,39.904' for coordinates). Anti-pattern: LLMs reuse example values literally in subsequent calls, causing failures. Baseline: 'Do not put example values in description; use constraints instead.'
Pagination is incomplete. Tools returning lists (#6, #7) accept current_page int but do not return total_count, next_cursor, or has_more flag. LLM cannot know when to stop paginating. Baseline: 'Tools returning lists should accept page/offset and return total count or next_cursor.'
Numeric parameters lack min/max bounds. image_num and video_num default to 3 with no stated maximum (could LLM pass 10000?). Baseline: 'Specify minimum and maximum for numeric parameters (e.g., page_size 1 - 100, days 1 - 365).'
Parameter descriptions are sparse (averaging 30-50 chars vs. baseline 72 chars). Many omit context on allowed values, format constraints, or dependencies. E.g., 'coordinates' in weather tool is described as List[object] without field documentation.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. Baseline: tools declare which are read-only, which modify state, which are safe to retry. Current code marks risks in metadata ('Risk: READ_ONLY', 'Risk: WRITE') but tool definitions contain no formal annotations for LLM consumption.