Loading ToolBench
A Model Context Protocol server for playing Diplomacy games with multiplayer support, game state management, chat, and order submission via WebSocket and JSON-RPC 2.0
Diplomacy MCP server has critical definition gaps. Tool descriptions are present but generic (10-50 chars, well below the 50-200 char LLM-optimized baseline). Input schemas exist but lack parameter descriptions entirely, every parameter is undescribed, violating the 100% A+ baseline. No output schemas documented. Error handling is minimal (raw error codes like -32010 with no recovery guidance). Tool naming is reasonable (verb_noun style: game.create, game.join, board.state) but parameter naming lacks type suffixes (e.g., 'code' should be 'game_code'). The 'initialize' tool is a vestigial MCP 1.x pattern (stateful handshake) that should be removed in favor of stateless request handling. No tool annotations (readOnlyHint, destructiveHint) despite clear WRITE vs READ_ONLY risk classifications. Composition is weak: chat.send and order.submit lack context about game state prerequisites.
Retrieve the current board state for a game
Send a chat message to other players in the game
Create a new Diplomacy game with optional name and map selection
Join an existing Diplomacy game with a player name and optional power assignment
Initialize the MCP server connection and retrieve server capabilities
Submit movement orders for the current phase
Mark the player as ready to advance to the next phase; advances phase when all players are ready
All parameters lack descriptions. Rubric baseline: 100% of A+ tools have parameter descriptions. Every parameter (name, code, power, message, orders, ready) is undescribed, forcing LLMs to guess intent.
No output schemas documented. Rubric baseline: 100% of A+ tools have documented return types. Callers cannot plan downstream tool chains or extract required fields (e.g., does game.create return game_id, code, or both?).
Tool descriptions are too short (10-50 chars vs. 50-200 char LLM-optimized baseline). Descriptions lack WHAT, WHEN, and prerequisites. E.g., 'board.state' says 'Retrieve the current board state' but does not explain when to call it or what fields it returns.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
Error handling lacks recovery guidance. Errors return raw codes (-32010, -32001) with minimal context. Rubric pattern: errors must tell LLM what to do next. E.g., '-32010: Join a game first' should be 'You must join a game before submitting orders. Call game.join() with a game code.'
Missing tool annotations. Tools have explicit WRITE/READ_ONLY risk classifications in metadata but no destructiveHint or readOnlyHint in schema. Agents cannot infer which tools are safe to retry or have side effects.