Loading ToolBench
Quality gate and risk control framework for LLMOps that exposes Gate0-Gate7 evaluation pipeline as MCP tools for AI agent trace assessment
Server has 7 tools with mixed quality. Tool names follow verb_noun pattern (evaluate_trace, check_tool_safety, get_gate_stats, health_check, web_search, file_read, shell_exec), good naming. However, descriptions vary widely in quality and completeness. Most critically, input schemas are visible but parameter descriptions are inconsistent. Several tools lack proper error handling guidance. The shell_exec tool has a whitelist but no validation error messages. Output schemas are not documented for any tool. No tool annotations (readOnlyHint/destructiveHint) present despite clear risk levels.
单次工具调用的 Gate0 前置安全检查。在实际调用工具前检查：工具是否已声明、风险等级、参数是否存在威胁。MCP agent 可以用此工具在每次 tool call 前做预检。
完整的 Gate0-Gate7 质量门评估。传入 agent trace（工具调用、LLM 决策、性能指标），返回各道门的通过/失败结果。用于在发布前判断一次 agent 运行是否可以进入生产。
读取指定路径的文件内容。仅支持文本文件，最大 100KB。返回文件内容或错误信息。
查询质量门仓库的历史评估统计。
检查 agent-prod 引擎和仓库健康状态。
执行安全的 shell 命令。允许的命令: cat, cut, date, df, du, echo, env, find, free, grep, head, hostname, id, ls, printenv, pwd, sort, stat, tail, tr, uname, uniq, uptime, wc, whoami。危险命令（rm/sudo/curl 等）将被拒绝。
搜索网页内容。返回相关网页的标题、URL 和摘要。适用于获取最新信息、技术文档查询等场景。
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract fields without knowing response structure.
shell_exec lacks actionable error messages. Whitelist enforcement exists but tool description does not explain what happens when a non-whitelisted command is passed. LLM cannot self-correct.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. shell_exec is WRITE; evaluate_trace, check_tool_safety, get_gate_stats, health_check, web_search, file_read are READ_ONLY. Annotations enable safer agent planning.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | <=2025-11-25 | v2 |
get_gate_stats and health_check have minimal descriptions (50 and 40 chars respectively). Descriptions must be 10 - 1024 chars and explain WHAT, WHEN, and WHY. Current descriptions lack context for LLM selection.
file_read accepts arbitrary paths with only a 100KB size limit. No validation against path traversal (e.g., '../../../etc/passwd'). Description does not mention security constraints or what happens on permission denied.