Loading ToolBench
25 tools with consistent verb-noun naming (get_, create_, update_, delete_, add_, remove_, archive_, search_). All tools have descriptions (avg ~80 chars, within baseline 34-392). Input schemas present with typed parameters and descriptions. However, output schemas are not documented in the source code, LLMs cannot see what fields are returned. Parameter descriptions lack format constraints (e.g., date format for 'due' param in create_card). No enum constraints for position parameters ('top', 'bottom', or number), free-form strings invite hallucinated values. Error handling exists (TrelloAPIError with mapped messages) but lacks recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Composition is sound, tools are single-responsibility and chainable (e.g., get_board returns lists, create_card accepts list_id). Missing: output schema documentation, parameter format constraints, error recovery hints, tool annotations.
Add a URL attachment to a card. Only URLs are supported (no file uploads).
Add a comment to a card.
Add a label to a card.
Archive (close) a card.
Archive (close) a list.
Close (archive) a board. This does not permanently delete it.
Create a new board. Set default_lists=false for a blank board.
Output schemas not documented. LLMs cannot see what fields are returned by tools (e.g., does get_board return lists, url, description?). This forces agents to guess and risks context loss when chaining tools.
Position parameters ('top', 'bottom', or number) lack enum constraints. Free-form strings invite hallucinated values like 'middle' or 'first'. Affects create_card, update_card, create_list, move_list.
Date format for 'due' parameter (create_card, update_card) not specified. LLMs may pass '2024-01-15', 'Jan 15', or epoch timestamps. Should document ISO 8601 format explicitly.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Create a new card on a list. Set position to "top", "bottom", or a positive float to control placement.
Create a new label on a board.
Create a new list on a board. Position can be 'top', 'bottom', or a number.
Delete an attachment from a card.
Delete a label from a board.
Get all attachments on a card.
Get a board's details including its lists.
List all boards accessible to you.
Get full details for a card including comments.
Get all cards in a list.
Get all labels defined on a board.
Get all open lists on a board.
Move a list to a new position. Use 'top', 'bottom', or a number.
Remove a label from a card.
Search for cards across boards. Optionally scope to a specific board.
Update a board's name or description.
Update a card's fields. Set list_id to move the card to a different list. Set position to "top", "bottom", or a positive float to reorder within the list.
Rename a list.
Error handling returns mapped messages (401, 404, 429) but lacks recovery guidance. E.g., 'Not found' does not suggest 'Try search_cards() to find the card ID' or 'Check if the board was archived'.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Agents cannot distinguish safe reads from destructive writes without parsing descriptions.