Loading ToolBench
MCP server that exposes annotation tools to external agents for multimodal annotation workflows
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Tool names follow verb_noun convention (start_*, search_*, export_*), which is good. However, descriptions are minimal (10-50 chars), parameter descriptions lack detail on constraints and formats, and output schemas are not documented. No error handling guidance visible. Naming is clear but descriptions fall well below the 50-200 char LLM-optimized baseline. Parameters lack type constraints (e.g., no enum for format in export_dataset despite having one in schema). No evidence of idempotency, pagination, or composition patterns.
Export annotations in a specified format (coco, yolo, voc, or ls_json)
Search annotations in a project by label, split, or source
Start a new annotation project with the given name and source folder path
Descriptions are too brief (10-50 chars) to guide LLM tool selection. Baseline for A+ tools is 50-200 chars. 'Start a new annotation project with the given name and source folder path' lacks context on when to use this vs other tools, what happens to existing projects, or what the return value contains.
Output schemas are not documented. LLMs cannot plan downstream calls or extract required fields (e.g., does start_annotation_project return a project_id? Does search_annotations return annotation IDs needed for export_dataset?). This breaks tool composition.
Parameter descriptions lack constraint details. 'Absolute path to source folder containing images' does not specify: must the folder exist? What image formats are supported? What is the max folder size? 'Filter by label name (optional)' does not explain what happens if the label does not exist.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2026-07-28+ | v2 |
No error handling guidance. If start_annotation_project fails (e.g., folder not found, permission denied), the LLM has no recovery path. No indication of which errors are retryable vs user-fixable.
export_dataset has an enum constraint in schema (coco, yolo, voc, ls_json) but the description does not mention it. LLMs cannot read JSON Schema, they rely on description text to understand valid options. This risks hallucinated format values.