Loading ToolBench
FastMCP provides 18 tools across multiple example servers with explicit schemas and descriptions. However, quality is inconsistent. Naming follows verb_noun convention (good), but many descriptions are terse (10-50 chars, below the 50-200 char baseline for LLM optimization). Parameter descriptions are present but often minimal. Schemas are properly typed JSON Schema, but output schemas are not documented. Error handling is basic, no recovery guidance or actionable error messages. The bulk_tool_caller tools are well-structured, but atproto and smart_home examples lack depth in parameter constraints and error context.
Activates a specific scene within a specified light group, verifying the scene belongs to the group.
Call a single tool registered on this MCP server multiple times with a single request. Each call can include different arguments. Useful for speeding up what would otherwise take several individual tool calls.
Call multiple tools registered on this MCP server in a single request. Each call can be for a different tool and can include different arguments. Useful for speeding up what would otherwise take several individual tool calls.
Create a thread of posts with automatic linking. The first post becomes the root of the thread, and each subsequent post replies to the previous one, maintaining the thread structure.
Echo a message, with debug info if debug mode is enabled.
Follow a user by their handle.
Minimal descriptions on 8 tools (like, repost, list_groups, list_scenes, get_status, follow, read_all_lights). Descriptions under 50 chars lack context for LLM tool selection. Baseline is 50-200 chars.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields without knowing response structure. All 18 tools lack return type documentation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2025-06-18+ | v2 |
Get the current server configuration and status.
Like a post by its AT URI.
Lists the names of all available Hue light groups.
Lists Hue scenes, grouped by the light group they belong to.
Create a post with optional rich features like images, quotes, replies, and rich text.
Lists the names of all available Hue lights using phue2.
Repost a post by its AT URI.
Search for posts containing specific text.
Sets the brightness of a specific light (0-254) using phue2.
Sets multiple attributes for all lights within a specific group.
Sets multiple attributes (e.g., hue, sat, bri, ct, xy, transitiontime) for a specific light.
Turns a specific light on (true) or off (false) using phue2.
No error handling guidance. Tools do not indicate what errors are retryable, user-fixable, or fatal. No recovery suggestions (e.g., 'User not found. Try search_users() first'). Affects all 18 tools.
Destructive tools (post, follow, like, repost, toggle_light, set_brightness, activate_scene, set_light_attributes, set_group_attributes) lack confirmation or dry-run support. No idempotent hints or destructive markers in tool definitions.
Parameter descriptions are minimal or missing context. E.g., 'handle' in follow tool lacks format guidance (should note 'e.g., user.bsky.social'). 'light_name' in toggle_light lacks guidance on how to discover valid names.