Loading ToolBench
FastAPI backend for a multi-LLM chat application with MCP (Model Context Protocol) support, file extraction, and settings management
This server has severe definition quality gaps. Tool names lack action verbs (handle_pdf, handle_html, handle_markdown, handle_jupyter, handle_generic, process_file are all vague). Descriptions are minimal (10-50 chars, below the 194-char baseline). Input schemas show UploadFile types but lack proper JSON Schema definitions with constraints. The handle_tool_call tool accepts a PolyMCPClient parameter, which is a code object, not a serializable input, this violates the tool parameter contract. No output schemas are documented. Error handling is absent from descriptions. No parameter constraints, enums, or validation guidance. Tool composition is poor: handle_* tools are near-duplicates with different file types, suggesting they should be unified or parameterized.
Extract text from generic files using unstructured
Extract text from HTML files
Extract text from Jupyter notebook files
Extract text from Markdown files
Extract text from PDF files
Handle a tool call by executing either a local function or an MCP server tool
Main entry point for processing files. Routes to appropriate handler based on file type.
Tool names lack action verbs. 'handle_pdf', 'handle_html', 'handle_markdown', 'handle_jupyter', 'handle_generic' use the generic verb 'handle' which does not convey the specific action (extract, parse, convert). LLMs cannot infer intent from these names.
Descriptions are too short (10-50 chars, well below 194-char baseline). 'Extract text from PDF files' lacks context on when to use this vs process_file, what format the output is, or what happens on error.
Input schemas are incomplete. UploadFile type is not a JSON Schema primitive, no maxSize, mimeType constraints, or format specification. Schemas lack proper JSON Schema structure with type, description, and constraints.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 20 | <=2025-11-25 | v2 |
handle_tool_call accepts a PolyMCPClient parameter, which is a Python code object, not a serializable JSON value. Tool parameters must be JSON-serializable primitives or objects. This violates the MCP tool contract.
No output schemas documented. LLMs cannot plan downstream calls or extract results without knowing what fields are returned. No pagination, error recovery guidance, or field descriptions.