Loading ToolBench
A MCP server definition to control a blocks world simulation
The server defines 8 tools with full descriptions and basic parameter schemas. However, several critical quality gaps prevent a higher score: (1) Parameter descriptions are present but minimal, most are one-line reiterations of the parameter name with no context on constraints, formats, or valid ranges. (2) Output schemas are undocumented, tools return strings or unstructured JSON without declaring what fields agents should expect. (3) Error handling is generic ('Failed to X') with no guidance on recovery steps. (4) The verify_plan and execute_plan tools use complex nested oneOf schemas with hardcoded enums for 26 blocks (A-Z), making the schema brittle and unmaintainable if block sets change. (5) Several tool names could be clearer: pick_up is non-standard (get_block or grasp_block would be more verb-forward); put_down is informal (place_block or release_block is clearer). (6) No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear write vs read-only distinctions. On the positive side: tool descriptions are detailed (200-400 chars), including preconditions and effects, which aids LLM planning. Naming is generally verb-forward. Schema presence is consistent across all tools. Parameter types are declared (string, array). The description depth shows thought and care.
Executes a given plan in the blocksworld simulation.
The action pick_up(x) represents picking up block x from the ground. Preconditions: - Block x is directly on the ground (i.e., not stacked on any other block) - Block x is clear (i.e., no block is on top of block x) - The robot is idle and empty (i.e., not holding any block) Effects: - The robot is now holding block x - Block x is no longer on the ground - Block x is no longer clear because it is being held - The robot is no longer idle Args: - block (str): The name of the block to pick up. Returns: - A string indicating the result of the pick_up(x) action, or an error message if the action fails.
The action put_down(x) represents placing block x from the robot onto the ground. Preconditions: - The robot is holding block x - A free stack must be available for placement (limited number of stack positions) Effects: - Block x is now on the ground as a new stack (i.e., not stacked on any other block and a free stack position is now occupied) - Block x is now clear (i.e., no block is on top of it) - The robot is no longer holding block x - The robot is now idle and empty Args: - block (str): The name of the block to be placed on the ground
Complete description of the blocksworld environment, rules, and available actions.
The action stack(x, y) represents placing block x from the robot onto block y. Preconditions: - The robot is holding block x - Block y is clear (i.e., no block is on top of block y, and block y may be on the ground or on another block) Effects: - Block x is now on top of block y (block y remains in its original position) - Block x is now clear (i.e., no block is on top of it) - Block y is no longer clear - The robot is no longer holding block x - The robot is now idle and empty Args: - block_x (str): The name of the block to be placed on top of block_y - block_y (str): The name of the block on which block_x will be stacked
Output schemas undocumented. Tools return unstructured strings or raw JSON objects without declaring expected fields, types, or structure. LLMs cannot plan downstream tool chains or extract required data reliably.
Parameter descriptions lack constraint metadata. E.g., 'block' params accept only A-Z but no min/max or regex specified in descriptions. 'The name of the block to pick up' does not hint that only single uppercase letters are valid. Forces LLM to guess or try invalid values.
Hardcoded block enum (A-Z) in verify_plan and execute_plan schemas. If the simulation ever needs to support more blocks or uses different naming (e.g., block_1, block_2), the schema breaks. Dynamic enum or external definition would be more maintainable.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 55 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 39 | - | v1 |
Queries the current status of the system, including the position of all blocks, held blocks by the robot and available positions on the ground. Returns: - A JSON object with the following structure: - robot.held_block: The name of the block currently held by the robot, or null if none - robot.state: The current status of the robot (e.g., 'idle') - stacks: Array of stack objects, each containing: - number: The stack number/position - blocks: Array of blocks in the stack from bottom to top, each with: - name: The block name (e.g., 'A', 'B', 'C') - x_size: Block width in units - y_size: Block height in units - position: Integer indicating vertical height in stack (0 = bottom block on ground, 1 = first block above ground, 2 = second block above ground, etc.)
The action unstack(x, y) represents unstacking block x from block y. Preconditions: - Block x is stacked directly on top of block y (i.e., x is not on the ground, and block y may be on the ground or on another block) - Block x is clear (i.e., no block is on top of block x) - The robot is idle and empty (i.e., not holding any block) Effects: - The robot is now holding block x - Block x is no longer on top of block y (block y remains in its original position) - Block x is no longer clear because it is being held - Block y is now clear - The robot is no longer idle Args: - block_x (str): The name of the block to be removed from the stack - block_y (str): The name of the block underneath block_x
Verify a given plan against the defined rules of the simulation. Does not execute the plan.
Error handling generic and non-actionable. All tools catch exceptions with 'Failed to X: ' + str(e). No guidance on retryability, whether to ask user, or next steps. Pattern:recovery-guide missing.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Status and rules are clearly read-only; execute_plan is destructive; verify_plan is idempotent. Annotations would help agents reason about safety and retry logic.
'put_down' is informal naming. More verb-forward conventions like 'place_block' or 'release_block' align with actionable verb_noun pattern. 'put_down' is colloquial; a blocksworld domain might use 'place' for clarity.
'rules' tool has vague description: 'Complete description of the blocksworld environment, rules, and available actions.' No mention of what structure it returns or when to call it (e.g., 'Call this first to learn valid actions and preconditions'). Return type undocumented.
Parameter block_x and block_y in unstack/stack are not self-documenting. Both named 'block_X' but semantic difference (one removed, one target) not obvious from name. Rename to 'block_to_move' vs 'block_target' for clarity.