Loading ToolBench
The QWeather MCP server has moderate definition quality with mixed results across tools. Tool naming is reasonably clear with action verbs (lookup_, get_, generate_), but descriptions vary significantly in quality and length. Some descriptions are excessively long with technical implementation details rather than user-facing guidance. Input schemas are present but basic, lacking enum constraints and detailed parameter descriptions. Parameter descriptions are minimal or missing entirely. Output schemas are not documented for any tool, forcing LLMs to infer response structure. Error handling is present in code but not surfaced in tool definitions. The server lacks proper composition patterns, jwt generation is exposed as a tool despite being an infrastructure concern that should be hidden behind service-side auth.
生成JWT令牌的API端点 参数: request (JWTRequest): JWT请求对象,包含过期时间等参数 返回: dict: 包含以下字段的响应字典 - jwt: 生成的JWT令牌 - expires_at: 令牌过期时间(ISO格式) - issued_at: 令牌签发时间(ISO格式) - valid_for_seconds: 令牌有效期(秒) 异常: HTTPException: 生成JWT过程中发生错误时抛出500错误 说明: - 使用EdDSA算法进行签名 - 默认有效期为15分钟 - 支持添加自定义声明
一站式查询城市天气 - 只需提供城市名 - 自动查询位置ID并获取天气
获取指定位置的实时天气 - 需要提供位置ID - 返回当前天气详情
根据城市名称查询位置ID - 返回城市的详细信息和位置ID - 位置ID用于后续天气查询
Output schemas are not documented. LLMs cannot infer what fields are returned by lookup_city (presumably location details, coordinates), get_weather_now (weather fields), or get_weather_by_city. This forces agents to guess and wastes tokens parsing unstructured responses.
Parameter descriptions are missing or trivial. lookup_city accepts 'location' with only '城市名称，如：北京' (city name, e.g. Beijing). No guidance on format, length, language support, or fallback behavior if name is ambiguous. get_weather_now accepts 'location' as 'location ID, e.g. 101010100' but does not explain how to obtain it or what happens if the ID is invalid.
generate_jwt is exposed as a tool but belongs in server-side authentication. JWT generation is an infrastructure concern, the MCP spec requires Bearer token auth, not agent-initiated token creation. This tool violates the secret-injection pattern: agents should never need to request tokens, and token generation should be transparent to the agent. The tool definition also leaks implementation details (EdDSA, iat/exp fields, key_id headers) that belong in docs, not prompt engineering.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 8 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 41 | - | v1 |
Tool descriptions are written in Chinese with mixed English, making them inaccessible to non-Chinese-speaking developers and LLMs optimized for English. Description for generate_jwt contains 376 characters of technical implementation details ('使用EdDSA算法进行签名', 'iat', 'exp', 'sub', 'kid', '提前30秒', '提前1分钟') rather than agent-facing guidance on when and why to call it.
No enum or constraint definitions. All parameters accept free-form strings. lookup_city 'location' and get_weather_now 'location' have no validation against known values, format patterns, or length bounds. get_weather_by_city 'city' likewise accepts any string. This invites hallucinated city names and invalid IDs.
Error handling is not surfaced in tool definitions. fetch_weather_api() in the code catches 401/403 and retries, but tools do not declare error conditions, recovery hints, or actionable messages. An LLM that receives 'Invalid city name' has no guidance on how to recover.
Tool composition is poor. lookup_city and get_weather_now are separate tools, forcing agents to chain them. get_weather_by_city combines the lookup and fetch in one call, which is better, but forces duplication. The tools should be designed so agents can either use the high-level get_weather_by_city (single call) or split into lookup_city + get_weather_now if they need intermediate location data. Current design unclear when each is preferred.