Loading ToolBench
CLI and MCP server for document ingestion with Docling.
Server has 5 tools with basic schemas and descriptions, but significant gaps limit production readiness. All tools have descriptions (10-80 chars, mostly short), and input schemas are present with type definitions. However, descriptions are sparse and lack LLM-optimized guidance on WHEN to use each tool or WHAT happens (especially for write operations). Parameter descriptions are minimal. Output schemas are not documented. Error handling is absent, no guidance on recovery, retryability, or what to do when files are missing. No tool annotations (destructiveHint, readOnlyHint, idempotentHint). The two write tools (ingest_document, batch_ingest) do not declare their side effects clearly. Tool composition is reasonable (separate read/write concerns), but descriptions do not explain dependencies or prerequisites.
Ingest multiple documents from a directory recursively.
Export the markdown content from an output directory.
Read and return the manifest.json file from an output directory.
Ingest a single document file and extract its content into structured markdown with manifest.
Read and return all chunks from chunks.jsonl file in an output directory.
Write operations lack side-effect declarations. ingest_document and batch_ingest modify the filesystem but descriptions do not state 'This tool creates files and directories' or warn about idempotency. LLMs cannot determine if retrying is safe.
No output schemas documented. Tools return dicts/lists but LLMs do not know what fields to expect. E.g., ingest_document returns result.model_dump(), what fields? batch_ingest returns {results: [...]}, but what is in each result? This forces LLMs to guess and breaks downstream tool chaining.
Parameter descriptions are minimal or missing. 'backend' param has description 'Backend to use: auto, fake, or docling' but does not explain when to use each or what 'auto' does. 'source' and 'source_dir' lack guidance on path format (absolute vs relative, must exist?). 'out' is optional but no guidance on default behavior.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
No error handling or recovery guidance. If a file is not found, API fails, or output directory is missing, tools return raw exceptions. No actionable error messages. E.g., 'File not found: /path/to/doc.pdf. Ensure the path is absolute and the file exists.' would help LLMs self-correct.
Tool descriptions are too short (30-80 chars). Baseline for A+ tools is 194 chars. Current descriptions lack context on WHEN to use each tool, dependencies, and prerequisites. E.g., 'Ingest a single document file and extract its content into structured markdown with manifest.' does not explain that you must call ingest_document before calling get_manifest or read_chunks.