Loading ToolBench
Single tool 'sympy' has a lengthy description (500+ chars, exceeds 200-char LLM-optimized baseline) mixing rules, examples, and error guidance into one block. Input schema visible but minimal: only 'code' parameter with basic string type and description in Chinese. No output schema documented. Tool performs code execution (destructive/risky operation) but lacks error classification, recovery guidance, or confirmation pattern. Description is verbose and tutorial-like rather than concise and action-oriented. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite WRITE risk classification.
Run Python/SymPy code in a sandbox and return compact JSON. Rules: input is `code`; print final answers; only `sympy`/`math` imports are allowed; file/system/network/dynamic execution are blocked. Use SymPy functions (`log(x)`, not `symbols("ln")(x)`). For numeric answers, assign all constants and inspect `expr.free_symbols` if symbols remain. Data pitfalls: `zip(...)` truncates to the shortest input; avoid adding labels when strings already contain "A."; this executes code only. One-shot: `import sympy as sp x=sp.Symbol("x") expr=sp.integrate(sp.sin(x), (x, 0, sp.pi)) print(expr) print(sp.N(expr))` Errors include `code`, `line`, `err`, `hint`; inspect "可能原因" in `err`.
Input schema incomplete: only 'code' parameter defined; no output schema documented. LLM cannot infer response structure or plan downstream operations.
Description exceeds 200-char LLM-optimized baseline (500+ chars) and mixes rules, examples, and error guidance. Verbose format wastes tokens and buries key intent. Should state: WHAT (execute SymPy code), WHEN (math/symbolic computation), WHAT IT RETURNS (JSON with output or error).
Tool marked WRITE risk but lacks destructiveHint annotation, error classification (retryable vs fatal), or confirmation pattern. Agents cannot determine if code execution is safe to retry or requires user approval.
Parameter 'code' description is in Chinese ('包含 Python/SymPy 逻辑并通过 print 输出结果') and lacks format constraints. No mention of allowed imports, forbidden operations, or expected output format. LLM cannot validate input before submission.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | <=2025-11-25 | v2 |
Error responses include 'code', 'line', 'err', 'hint' fields but no recovery guidance. 'hint' field is vague, does not tell LLM whether to retry, ask user, or abandon. Missing error classification pattern.