Loading ToolBench
Hydra MCP Server has 4 tools with explicit registration (hydra_mcp/tools.py). Tool descriptions are present but brief (10 - 80 chars, well below the 194-char median). Input schemas are properly defined with types and descriptions, but output schemas are completely undocumented, LLMs cannot predict return structure. Error handling uses a custom MCPError dataclass with structured responses (isError flag, structuredContent, TextContent), which is good, but error messages lack actionable recovery guidance. Naming is clear and verb-driven (hydra*), but descriptions are minimal, and the server does not implement tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite two tools being WRITE operations.
Return information about the Hydra implementation used by this MCP server.
Initialize Hydra inside a git project. Validates and sets `projectRoot`, runs: `hydra init [--session <name>]`, optionally installs `hydra.py` into the project root (symlink/copy).
Run an arbitrary `hydra.py` CLI command inside a project. Example: args=["task","list"]
Set (and validate) the active project root for subsequent tool calls.
Output schemas are completely undocumented. Tools return CallToolResult with structuredContent (dict) and TextContent (JSON), but LLMs cannot predict the fields, types, or structure of the result object. Example: hydraRun returns a dict with 'code', 'out', 'err', this is inferred from code, not declared in a schema.
Tool descriptions are too brief (10 - 50 chars). 'Return information about the Hydra implementation...' (50 chars) and 'Set (and validate) the active project root...' (45 chars) lack context on when to use these tools, expected inputs/outputs, and error cases. Median production tool description is 194 chars. Descriptions under 20 chars scored 0-20; these at 10 - 50 range are scored 35 - 50, indicating insufficient detail.
Tool annotations missing. hydraInit and hydraRun are WRITE operations (create, modify state), but the MCP registration does not include readOnlyHint/destructiveHint/idempotentHint flags. LLMs cannot distinguish safe (READ_ONLY) vs. dangerous (WRITE) tools without explicit annotations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2025-06-18+ | v2 |
Error messages lack actionable recovery guidance. MCPError includes a code and message, but messages like 'Not a git repository: {path}' or 'hydra.py not found next to hydra_mcp package' do not tell the LLM what to do next or suggest alternative actions (e.g., 'Ensure the path is inside a git repository' or 'Check that hydra.py is installed').
Parameter descriptions are minimal. 'Absolute path to the project root directory' (43 chars) is sufficient but lacks format constraints (must be absolute, must be a directory, must be a git repo). Descriptions should state: 'Absolute path to the project root directory. Must exist and be a git repository.'
No pagination or result limits documented for hydraRun output. The tool captures stdout/stderr as strings; if hydra.py produces very large output (e.g., verbose logs), this could exceed context limits. No limit or truncation is mentioned.
No dry-run or confirmation support for destructive tools. hydraInit and hydraRun modify project state (initialize hydra, install hydra.py, run commands). No confirmation or preview mechanism exists to prevent accidental operations.