Loading ToolBench
MCP server for Hypernode management with tools for attack blocking, incident management, nginx log analysis, and shell command execution
This server exhibits significant quality gaps across naming, descriptions, parameters, and schemas. Of 8 tools, most lack proper parameter descriptions, and parameter schemas are incomplete or missing entirely. Tool names follow a verb_noun pattern (block_attack, list_attacks, get_incident, etc.), which is positive, but descriptions are often generic and fail to guide LLM decision-making. Parameter documentation is the most critical failure: many parameters lack descriptions, and type constraints are minimal. The server has no visible output schema documentation, and error handling is absent from tool definitions. The auto-registration pattern in tools/__init__.py and generic.py suggests tools are registered, but the code snippet does not show the actual schema definitions passed to FastMCP.run() or the @mcp.tool() decorators, only the tool class definitions are visible. This limits confidence in actual schema completeness.
Analyze nginx logs with optional filters.
List available fields for nginx log analysis using pnl --list-fields.
Block a specific attack type using hypernode-systemctl block_attack.
Execute a shell command safely (dangerous commands are blocked).
Get files from a specific incident directory.
Simple hello world tool for testing.
List all available but not necessarily enabled known attack-blocking options on the Hypernode.
List all incidents in the ~/incidents directory.
Parameter descriptions missing or incomplete. Most parameters lack descriptions or have vague descriptions (e.g., 'The incident date/directory name' for get_incident's date parameter). Parameters like file_pattern in get_incident have no description at all.
No output schemas documented. Tools return objects (success, result, attacks, etc.) but the LLM has no advance knowledge of the structure. LLM must infer from tool behavior, increasing reasoning cost and error likelihood.
Enum constraints missing for constrained parameters. block_attack's attack_type should be an enum or reference list_attacks() output. analyze_nginx_logs' filter and unique_by_field accept free-form strings without validation, inviting hallucinated values.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 36 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 35 | - | v1 |
No error handling guidance. Tools define no error recovery paths, actionable error messages, or retry strategies. execute_shell_command (WRITE risk) offers no confirmation or dry-run capability.
Tool descriptions lack decision-making context. Most descriptions state WHAT the tool does but not WHEN to call it vs. similar tools or WHAT the common failure modes are. Example: 'List all incidents' tells LLM nothing about incident structure or why it should call this before get_incident.
Parameter format/constraint documentation missing or informal. get_incident's 'date' is ambiguous (is it YYYY-MM-DD or a directory name?). analyze_nginx_logs' filter uses example syntax in description ('Format: <field>=<str>') instead of a formal regex pattern. No min/max bounds on numeric parameters.
Destructive tool (block_attack, execute_shell_command) lacks confirmation/dry-run pattern. No mechanism to preview what will happen before committing. Agent mistakes (e.g., blocking the wrong attack type) have irreversible consequences.
Tool auto-registration pattern (tools/__init__.py, tools/generic.py) is not fully visible in source. Cannot verify that schemas are correctly passed to FastMCP or that tool names match the method names. Inferred tool definitions cap scoring.
Discovery tool dependencies not documented. analyze_nginx_logs references valid field names but does not link to analyze_nginx_logs_fields. LLM may pass invalid field values without knowing to call the discovery tool first.
hello_world is a testing artifact in production. Generic tool name does not follow verb_noun pattern. Adds noise to tool catalog and wastes LLM reasoning on a useless tool.