Loading ToolBench
A Model Context Protocol (MCP) server platform for Apache OFBiz that enables dynamic tool registration and execution with authentication support
Single tool 'findProductById' has a reasonable name and schema, but the description is extremely terse (49 chars) and lacks critical context. The parameter description mentions constraints (2-20 chars) but doesn't explain WHEN to use the tool, WHAT it returns, or WHAT happens on failure. No output schema is documented. The tool loader dynamically imports tools from disk, but actual tool implementation file is not provided in the source, only the examples/tools/src/findProductById.ts filename is referenced, meaning the actual ToolDefinition registration cannot be verified. Per hard scoring rule: 'If you cannot see the actual tool definition in the source (only inferred): cap that tool's overall at 50.' The schema is visible in the TOOLS section metadata, but without seeing the implementation file, we cannot confirm it matches the actual export.
Find a product by using its ID. If the ID is not provided, ask for it.
Tool description is critically undersized (49 chars). Rubric baseline: 194 chars average, p10=34, p90=392. This description is below p10 and violates the requirement that descriptions be 10-1024 chars with sufficient context for LLM selection. Current: 'Find a product by using its ID. If the ID is not provided, ask for it.' Missing: WHEN to use it, WHAT it returns, error handling guidance.
No output schema documented. The tool definition shows input schema but nowhere in the provided source is the output structure defined. LLMs cannot plan downstream operations without knowing what fields the tool returns. Violates: 'Document the output schema. LLMs need to know what fields to expect so they can plan downstream tool calls.'
Tool implementation file not visible in provided source code. Only the filename 'examples/tools/src/findProductById.ts' is referenced. The actual ToolDefinition registration, return type, and error handling are not verifiable from the source provided. Per hard scoring rule: tool definitions inferred rather than directly visible are capped at 50.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 41 | - | v1 |
Parameter description lacks actionable constraints. Current: 'ID of the product to find; must be between 2 and 20 characters long.' Missing: format guidance (alphanumeric? slug? UUID?), example behavior if ID not found, relationship to other tools (are there bulk lookup tools?), how to discover valid product IDs.
No error handling guidance. The description says 'If the ID is not provided, ask for it' but provides no guidance for: product not found, invalid ID format, API timeouts, or permission errors. LLM cannot decide whether to retry, ask the user, or abandon.