Loading ToolBench
MCP server for document operations (Word, Excel, PDF)
Server has 5 tools with basic verb-noun naming (create_word_document, edit_word_document, etc.). All tools have descriptions (50-100 chars, below the 194-char baseline). Input schemas are present with type declarations and parameter descriptions. However, critical gaps exist: no output schemas documented, no error handling guidance, no enum constraints on operation types, and descriptions lack WHEN/WHY context. The edit_word_document and edit_excel_file tools accept 'operations' arrays with undocumented structure (operation type must be inferred from code). No pagination, no result limits, no idempotency guarantees. Security concern: all tools perform file I/O without path validation or sanitization, risking directory traversal attacks.
Convert a text file to a Microsoft Word document.
Create a new Excel file with the provided content.
Create a new Microsoft Word document with the provided content.
Edit an existing Excel file using the specified operations.
Edit an existing Microsoft Word document using the specified operations.
Operations parameter lacks schema documentation. edit_word_document and edit_excel_file accept 'operations' as an array of dicts with type field, but valid operation types (add_paragraph, edit_paragraph, delete_paragraph, etc.) are not enumerated in the schema. LLMs cannot infer valid operation types without reading implementation code.
No output schemas documented. Tools return dicts with 'success', 'message', 'filepath' fields, but LLMs have no formal schema to parse responses or plan downstream calls. Missing documented return types violates baseline (100% of A+ tools have documented return types).
No path validation or sanitization. Tools accept 'filepath' and 'source_path' parameters without checking for directory traversal (../, absolute paths, symlinks). An LLM could be tricked into writing files outside intended directories.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | <=2025-11-25 | v2 |
Descriptions lack WHEN/WHY context. Tool descriptions state WHAT they do but not WHEN to use them or how they differ from similar tools. E.g., 'Create a new Microsoft Word document' does not explain when to use create_word_document vs convert_txt_to_word. Baseline: descriptions should answer 'What does it do? When should the LLM call it instead of a similar tool?'
No error recovery guidance. Error responses return 'success: false' with a message, but do not guide the LLM on next steps. E.g., 'File not found' does not suggest calling convert_txt_to_word first or checking the path. Violates pattern:recovery-guide.