Loading ToolBench
MCP server that lets AI agents escalate questions to humans through a markdown file interface
The ask_human tool has a clear purpose and acceptable naming, but suffers from moderate gaps in parameter descriptions, missing output schema documentation, and inadequate error handling guidance. The tool is functional but lacks production-grade polish. Descriptions are present but minimal (under 200 chars for main tool, which is acceptable at the lower end). Input parameters are defined with types but lack detailed constraint documentation. Output structure is not formally documented, the code returns JSON but LLMs have no schema reference. Error handling exists but provides minimal guidance on recovery strategies.
Ask a question to a human and wait for their response. The question is written to a markdown file that the human can edit with answers.
Output schema not documented. Tool returns JSON response but LLMs have no formal schema for what fields to expect or how to chain downstream tool calls. Code shows response structure but tool definition lacks returnType documentation.
Parameter descriptions lack constraint details. The 'timeout' parameter has no guidance on valid range, units, or server defaults. 'question' has no length guidance.
Error handling does not guide recovery. Code raises AskHumanError, QuestionTimeoutError, InputValidationError, and FileAccessError, but tool descriptions and error messages do not guide LLMs on when to retry, ask user, or escalate.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 48 | - | v1 |
No idempotency guarantee documented. Tool writes to a markdown file with UUID-based tracking, but description does not state whether repeated calls with same input return same result or generate duplicate entries.
Timeout behavior not clearly specified. Code has timeout logic with default 30 minutes, but tool description does not explain what happens when timeout expires, does it return an error, a null answer, or a partial result? LLMs cannot infer this.
Tool modifies state (writes markdown file) but description does not explicitly state this. Agents need to know which calls are safe to retry and which have irreversible consequences.'
No resource dependencies or prerequisites documented. The tool requires a markdown file at a specific path and file watching to detect responses. If the file path is misconfigured or file permissions denied, the tool fails silently or with opaque errors. Description should state prerequisites.