Loading ToolBench
MCP server for automatic documentation generation using OpenRouter
The server defines 3 tools with consistent schemas and moderate documentation. All tools follow a verb_noun naming pattern (generate_documentation, autoreview, autotestplan) which is correct. However, descriptions are brief (110-145 chars), parameter descriptions lack depth regarding constraints and valid values, and no output schemas are documented. The base-tool.ts class provides a solid foundation with getInputSchema() method, but the actual tool implementation lacks rigor in error handling guidance and recovery paths. Tools are write-heavy (WRITE risk on all 3), but descriptions do not explicitly warn about irreversible consequences. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present despite all being destructive operations.
Generates a code review for a repository by recursively analyzing directories and files, focusing on security issues, best practices, and potential improvements
Generates a test plan for code in a repository by recursively analyzing directories and files
Generates documentation for a code repository by recursively analyzing directories and files
Tool descriptions lack context about destructive consequences. All three tools perform file writes (WRITE risk), but descriptions do not state 'This tool modifies your codebase by writing files.' Agents need explicit knowledge that these calls are irreversible.
Parameter 'model' lacks constraint documentation. Description says 'defaults to Claude 3.7' but does not enumerate valid model options or format. LLM may hallucinate invalid model names. Should specify: 'Valid models: claude-3.7, gpt-4, ...' or link to OpenRouter API docs.
No output schema documented. Tools return AutoToolResult (per base-tool.ts) with fields: outputPath, success, content, error, isUpdate, skipped. But the MCP tool definitions visible in the assessment do not expose this schema to clients. Agents cannot plan downstream calls without knowing return structure.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 43 | - | v1 |
Parameter 'openRouterApiKey' stored in tool parameters. Code comment notes 'optional, can also be set in environment variable' but pattern:secret-injection forbids exposing credentials as parameters. Agents log all parameters, secrets leak into traces. Move to server-side env-var injection only; remove from parameter list.
Parameter 'updateExisting' has boolean type but description lacks clarity on behavior. Current text: 'Whether to update existing [tool] files (optional, defaults to true) or only create missing ones'. This is ambiguous, does 'update' mean append, overwrite, or merge? What happens if a file exists and updateExisting=false? Specify exact behavior.
No error handling guidance in tool descriptions. If OpenRouter API rate-limits or fails, what should the LLM do? Can it retry? Should it ask the user for a different model? Descriptions must include recovery paths (pattern:recovery-guide).
Tools lack idempotency guarantees. If an agent retries generate_documentation with the same path after a partial failure, will files be duplicated, overwritten, or merged? Code references 'isUpdate' flag but agent cannot predict behavior without explicit idempotency contract.