Loading ToolBench
Expose the Chessmata chess API to AI agents via MCP. Provides tools for authentication, game discovery, game management, and draw handling in a multiplayer chess platform.
Chessmata has 14 well-named tools with consistent verb_noun conventions (login, logout, get_*, list_*, create_, join_, make_move, resign_game). All tools have descriptions (avg ~90 chars) and documented input schemas with type definitions. However, descriptions are often minimal (10-40 chars for some), parameter constraints are informal (e.g., 'limit' lacks min/max specifications), and output schemas are not explicitly documented in the source. Error handling is present but generic. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite having clear READ_ONLY and WRITE/IRREVERSIBLE operations. Tools are well-composed (each does one thing) and accept natural identifiers (email, display_name) alongside system IDs. Parameter naming is consistent (session_id, player_id, user_id). The server lacks response shaping guidance for stripped/minimal output, and some parameter relationships are undocumented (e.g., 'promotion' parameter only applies to pawn promotion moves).
Create a new chess game. Returns sessionId and playerId. Share the sessionId for an opponent to join.
Get the currently authenticated user's profile including Elo rating and game stats.
Get full game state including board position (FEN), players, clocks, and draw offers.
Get the leaderboard sorted by Elo rating.
Get all moves for a game in order.
Get paginated game history for a user.
Join an existing game as the second player.
Output schemas not documented in source. Functions like _game_to_dict() and _move_to_dict() exist but no formal schema is exposed to the LLM in tool definitions. LLMs cannot plan downstream calls without knowing response structure.
Minimal parameter descriptions lack actionable constraints. 'limit' parameter in list_active_games lacks explicit min/max (1-50 is stated only in description text, not schema). 'inactive_mins' default is undefined. LLMs cannot validate inputs without formal constraints.
No tool annotations despite clear operation semantics. 'get_*' and 'list_*' tools should declare readOnlyHint=true. 'resign_game' should declare destructiveHint=true. 'create_game' should declare idempotentHint=false. These hints guide agent planning and safety.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 63 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 61 | - | v1 |
List currently active games that can be watched or joined.
List recently completed games.
Login to Chessmata with email and password. Stores credentials for subsequent tool calls.
Logout from Chessmata and clear stored credentials.
Look up a user by their display name. Returns userId, displayName, and eloRating.
Make a chess move in a game.
Resign from a game, forfeiting to the opponent.
Parameter dependency undocumented. 'make_move' has a 'promotion' parameter that only applies to pawn promotion moves. Without explicit documentation ('only required when moving a pawn to the 8th rank'), LLMs may omit it when necessary or pass it when invalid.
Generic error handling. _error() function returns raw APIError messages without recovery guidance. E.g., if a user lookup fails, the response does not suggest alternatives ('Did you mean: ...'). Pattern: recovery-guide.
Irreversible operations lack confirmation step. 'resign_game' and 'make_move' (in a live game) are irreversible but have no dry-run or confirmation mechanism. An agent bug could resign mid-game without warning.
Descriptions are terse (20-90 chars) and lack context on when to use each tool. 'get_current_user' (70 chars) does not explain when to call it vs lookup_user. Discovery tools (get_leaderboard) lack guidance on expected result size and pagination.