Loading ToolBench
Educational demonstration repository for LLM API usage patterns, error handling, structured outputs, context management, and tool use patterns in Python. Contains demo code for building AI agents with OpenAI API.
This is an educational repository demonstrating tool patterns, not a production MCP server. Tools have reasonable descriptions (Chinese, 50-150 chars) and basic schemas with enums for constrained inputs. However, critical gaps: no output schemas documented, no error handling guidance in descriptions, no tool annotations (readOnlyHint/destructiveHint), and no evidence of MCP protocol implementation. Tools are defined in plain Python functions with OpenAI client integration, not as MCP tool definitions. The code shows good internal practices (parameter validation, structured error returns) but does not expose these as MCP-compliant tools.
在防火墙中封锁指定 IP 地址。这是一个危险操作，仅在确认 IP 有威胁时使用
执行防火墙操作（封锁/解封/限流）。这是一个危险操作，只在确认存在真实威胁后才使用。必须提供详细的操作原因。
查询指定 IP 地址的威胁情报信息，包括风险等级、地理位置、关联标签等
查询 IP 地址在威胁情报数据库中的声誉评分和历史记录。当需要判断某个 IP 是否存在威胁时使用此工具。
在 SIEM 系统中搜索安全日志
搜索安全日志。当需要查找与特定 IP、用户或事件相关的日志时使用。
No output schemas documented. Tools return structured dicts but LLMs cannot see expected response fields, types, or pagination structure. Agents cannot plan downstream calls or extract data reliably.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Destructive tools like execute_firewall_action and block_ip lack explicit safety markers. LLMs cannot distinguish safe reads from dangerous writes without parsing descriptions.
Error handling descriptions lack recovery guidance. While internal code returns structured errors with suggestions, tool descriptions do not tell LLMs what to do on failure (retry, ask user, try alternative tool).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
Not an MCP server. Code is a standalone Python demo using OpenAI client directly, not implementing MCP protocol. No transport layer, no tool registration via MCP, no protocol handshake.
Duplicate/overlapping tools. search_security_logs and search_logs both search logs with nearly identical schemas. LLMs will waste reasoning cycles deciding between them. Consolidate into one canonical tool.