Loading ToolBench
Local MCP server exposing AkShare finance tools
Three tools with complete JSON schemas and basic descriptions. Naming is clear and verb-prefixed. However, descriptions are terse (9-16 words), lack context about use cases and failures, and parameters lack actionable constraints. Error handling is present in code but not surfaced in schemas. No output schema documentation visible. Parameter descriptions are minimal, e.g., 'Stock symbol' provides no guidance on format nuances. The server shows competent implementation but falls short of LLM-optimized tool design; descriptions should be 50-200 chars and guide selection logic.
Retrieve historical price data for a China A-share stock within a date range
Retrieve current quote for a China A-share stock symbol
Retrieve current snapshot of major China stock indices
Tool descriptions are extremely terse (9 - 16 words); lack guidance on when to use each tool vs alternatives, failure modes, or data sources. Descriptions should answer: WHAT, WHEN to call, and what HAPPENS. Current descriptions ('Retrieve current quote...') omit prerequisites, error recovery, and disambiguation between similar tools.
Parameter 'symbol' description ('Stock symbol (numeric or with prefix)...') is present but underdocumented: does not state acceptable formats clearly, does not show examples of valid/invalid inputs, and does not explain the zero-padding behavior. An LLM may pass '6' expecting it to work, only to fail. Add format constraint or regex pattern and example: 'Format: 6-digit code (e.g., "000001" for Shanghai 1) or "SH000001"/"SZ000001" with prefix. Auto-pads single-digit codes to 6 chars.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 39 | - | v1 |
Date parameters 'start_date' and 'end_date' description states 'YYYYMMDD format' but does not validate or document edge cases (e.g., invalid dates, start > end, weekend/holiday handling). Code validates format but error messages are not exposed to LLM. Add: 'ISO 8601 alternate (YYYY-MM-DD) also accepted. Invalid or out-of-range dates will error; try weekday dates to avoid holidays.'
Output schema is NOT documented in tool definitions. Code returns structured dicts with 'ok', 'source', 'data', and fallback fields, but LLM has no way to know what fields exist or their types. Document output: e.g., '{"ok": boolean, "source": string, "symbol": string, "data": [record], "error"?: string}' so agent can chain results.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all three tools being read-only. Lack of annotations means agent cannot reason about safety without reading descriptions. Add @readOnlyHint to all three tools to signal 'safe to retry'.
Error handling is robust in code (try-except fallbacks, validation) but NOT surfaced to LLM. Tool descriptions do not mention failure modes or recovery paths. E.g., get_a_share_quote falls back to daily data if realtime source fails, but LLM has no way to know this behavior exists. Document in description: 'Falls back to latest daily bar if realtime endpoint unavailable.'
Parameter 'limit' in get_china_index_snapshot and get_a_share_history accepted as integer with no bounds. Code calls max(limit, 0) to handle negatives, but description does not state range. Add: 'Integer, 1 - 500 (default 20). Limits result set to conserve tokens.'