Loading ToolBench
Educational framework demonstrating AI agent development with tool calling, RAG, web services, and multi-agent systems. Includes tool registry, LLM integration, and trace logging.
Educational server with 5 tools covering basic file I/O, math, and time operations. All tools have descriptions (10 - 200 chars, within baseline 34 - 392 range) and input schemas with typed parameters. Strengths: clear descriptions explain WHEN to use each tool (e.g., calculator warns against unit conversion); path traversal protection in read/write/delete; error messages guide recovery (e.g., 'file not found' lists available files). Weaknesses: no output schemas documented; descriptions lack dependency hints (e.g., 'read_file first to see structure before write_file'); no parameter constraints (e.g., max_chars lacks min/max bounds); error handling is basic (no retryable/fatal classification); write_file and delete_file lack confirmation UI integration despite requires_confirmation flag; no tool annotations (readOnlyHint, destructiveHint) in schema.
计算一个纯数学表达式并返回精确结果。当用户需要加减乘除、乘方等数值计算时使用。不要用它做单位换算、日期计算或字符串处理。
删除沙箱目录下的文件，不可恢复。仅当用户明确要求删除某个文件时使用。
获取当前日期和时间。当用户问「现在几点」「今天几号」，或需要以当前时间为基准计算时使用。它不是计时器，不能做倒计时。返回 YYYY-MM-DD HH:MM:SS 格式字符串。
读取沙箱目录下某个文件的内容。当用户想「看看文件里写了什么」时使用。它返回的是**文件内容**，需要文件名列表请直接说明，不要用本工具。
把内容写入沙箱目录下的文件（覆盖同名文件）。当用户明确要求「保存/记录下来」时使用。只写不改：要修改已有内容，先 read_file 看清楚再整体重写。
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract fields without knowing response structure.
Numeric parameter 'max_chars' in read_file lacks min/max bounds. Unbounded integers let LLMs pass absurd values (e.g., 999999999) that waste tokens or cause timeouts.
write_file and delete_file declare requires_confirmation=True but no MCP confirmation UI integration visible. Confirmation flag is ignored by the protocol handler.
Tool descriptions lack dependency hints. E.g., read_file should note 'Call this first to see file structure before write_file.' Multi-step workflows are not guided.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | <=2025-11-25 | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) in schemas. LLMs cannot distinguish safe vs. risky tools without reading descriptions.