Loading ToolBench
An MCP server for converting documents between multiple formats using Pandoc
The server implements a single document conversion tool with a complete JSON Schema, good descriptions, and basic error handling. However, it lacks several production-grade patterns: no output schema documentation, missing parameter constraints (enums for format), no pagination support (though not applicable here), weak error recovery guidance, and no idempotency hints. The tool description is solid (194 chars, within baseline 34-392 range) and parameters are typed with descriptions. The implementation shows competent error handling for file validation and Pandoc failures, but the tool interface itself could be more defensive about format validation and recovery suggestions.
Converts a document from one format to another using Pandoc. Args: input_file_path: The absolute or relative path to the input document file. output_file_path: The absolute or relative path where the converted output file should be saved. The directory will be created if it doesn't exist. to_format: The target format for the conversion (e.g., 'markdown', 'docx', 'pdf', 'html', 'rst', 'epub'). See pandoc documentation for full list. from_format: The format of the input file. If None, pandoc will try to guess from the file extension. Specify if the extension is ambiguous (e.g., 'md' for markdown). Defaults to None. extra_args: A list of additional command-line arguments to pass directly to pandoc (e.g., ['--toc'] for table of contents, ['-V', 'geometry:margin=1.5cm'] for PDF margins). Defaults to None. Returns: A string indicating success (including the output path) or an error message.
Output schema not documented. Tool description states 'Returns: A string indicating success...' but LLM has no structured representation of success/failure outcomes. Should document a structured response type (e.g. {success: bool, output_path: string, error?: string}).
No enum constraints on 'to_format' and 'from_format' parameters. Description says 'See pandoc documentation for full list' but LLM has no machine-readable valid values. LLM will hallucinate unsupported formats. Should define allowed values as enum or constraint.
Error messages return free-text strings without recovery guidance. 'Input file not found at <path>' is clear, but 'Error during conversion: <pandoc error>' buries Pandoc's stderr. Should categorize errors (retryable, user-fixable, fatal) and suggest next steps (e.g., 'Check file permissions' or 'Verify Pandoc supports this format').
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 53 | - | v1 |
No idempotency guarantee documented. Repeated calls with identical inputs should produce identical outputs, but source code does not declare this. Tool overwrites output file without warning, should document whether re-running is safe or if LLM should check file existence first.
No timeout or resource limit documented. 'timeout=30' in FastMCP config, but description does not warn LLM about conversion time for large files or complex formats. LLM has no expectations about execution duration.