Loading ToolBench
Generate, preserve, and verify lean repo context for AI coding agents
The server implements 7 tools with explicit schema definitions in mcp_server.py. All tools have descriptions (10-50+ chars) and documented input parameters with types. However, descriptions are uniformly terse (typically 8-15 words), falling short of the 50-200 char guidance for LLM-optimized tools. Output schemas are not documented in the tool definitions, validation functions exist but their structure is not visible in source. Parameter descriptions are minimal and generic ('Repository path', 'Token budget') without actionable guidance on constraints, ranges, or when to use each tool. No error handling patterns, recovery guidance, or tool-chaining context visible. Security considerations (path traversal, input validation) are not evident in the shown code. The tool set exhibits reasonable composition (status/check/detect/understand as read-only; init/update/pack as write operations), but lacks the richness expected for production agent interaction.
Check AGENTS.md documentation integrity and compliance
Detect repository structure and technology stack
Initialize AGENTS.md configuration and documentation for a repository
Generate LLMO pack files for LLM training/context optimization
Get status of repository AGENTS.md documentation
Build understanding payload with repository context and AI-optimized documentation
Update AGENTS.md and related documentation with latest repository context
Tool descriptions are uniformly terse (8 - 15 words). Best practice is 50 - 200 chars with WHAT the tool does, WHEN to use it, and WHO should call it. Current descriptions like 'Get status of repository AGENTS.md documentation' and 'Check AGENTS.md documentation integrity and compliance' lack context for tool selection. LLMs cannot distinguish between similar tools or understand when to prioritize one over another.
Parameter descriptions are minimal and generic. For example, 'Repository path' and 'Token budget for compact output' lack constraint guidance. No min/max ranges, no format specifications, no examples of valid values. The 'compact_budget_tokens' parameter has a default of 4000 but no upper bound documented. The 'llm_timeout_seconds' defaults to 30 with no guidance on reasonable ranges (1 - 300?). LLMs will pass arbitrary values without bounds enforcement.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Output schemas are not documented. The code contains validation functions (validate_mcp_status_response_payload, validate_mcp_check_response_payload, etc.) but the structure and fields of these responses are not visible in the provided source. LLMs cannot plan downstream tool calls or extract the right data without knowing what fields to expect. This violates the 100% baseline for A+ tools.
No error handling guidance. Error patterns ('recovery-guide', 'error-classification') are not visible. When tools fail, LLMs will not know: is it retryable? Should I ask the user? Is it fatal? No actionable error messages with constraints violated or alternatives offered. A bare 'path not found' tells the agent nothing about recovery steps.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). The Risk metadata in the spec lists some tools as 'WRITE' operations (init, update, pack) but this is not encoded in the tool schema or description. LLMs cannot automatically infer which tools are safe to retry, which require confirmation, or which might have side effects.
No tool composition context. Tools like 'understand' and 'detect' appear related but their interdependencies, call sequences, and data flow are undocumented. An agent does not know: do I call detect first, then understand? Can I skip detect? What IDs or references does understand need from detect?
Unclear tool boundaries and responsibilities. The tools 'check' and 'status' both relate to repository validation/state. The 'init', 'update', and 'pack' operations all modify state but without clear guidance on what each does differently or in what order they should be called. This violates the 'one tool, one job' principle.
String parameters ('path', 'stack', 'preset', 'llm_provider', 'llm_model', 'llms_format', 'site') lack enums or valid-value guidance. For example, 'preset' has a default of '' with no description of valid presets. 'llm_provider' defaults to '' but tools expect 'openai' or 'anthropic', LLMs will hallucinate invalid provider names without enum constraints.
Path parameter handling does not enforce safety. All tools accept a 'path' parameter but there is no visible validation against path traversal (/../../../etc/passwd). Given that these are filesystem operations, malicious or confused input could read/write outside intended directories. No sanitization or allowlist pattern visible.