Loading ToolBench
This Connect4 MCP server exhibits significant definition quality gaps. Both tools have descriptions present but lack depth and actionability. Tool naming follows basic verb_noun convention (play_connect4, move_connect4) but descriptions are vague about tool behavior, game state, and prerequisites. The play_connect4 tool has an empty input schema ({}), violating the principle that tools should clearly document their contract. move_connect4 has a single integer parameter 'col' with description present but no documented range constraints, validation rules, or error handling guidance. Neither tool describes output structure or error recovery paths. The server uses stateful game management via lifespan context but does not document this architectural pattern in tool descriptions, forcing LLMs to infer game flow. No tool provides guidance on expected agent invocation patterns, prerequisites, or multi-step dependencies. Descriptions average ~70 chars (below the 194-char baseline for A+ tools) and lack the prompt-engineering clarity needed for reliable LLM selection and invocation.
Plays a move of connect 4 if is legal. This tool is to be used if the assistant wants to play, or to play the move of the user, by the assistant.
Starts a game of connect 4, one should be polite and ask who plays first. Can also restart game...
play_connect4 has empty input schema ({}). Per HARD SCORING RULES, tools with no input schema must receive schema score of 0. This violates pattern:tool which requires all tools to document their contract with explicit schemas.
move_connect4 parameter 'col' lacks documented constraints. Description says '0 to 6' but no formal range validation (min/max), no enum, no length restriction in schema. Parameter description is only 36 chars, below the 72-char baseline. LLMs cannot reliably infer valid values without formal constraints.
Both tool descriptions lack WHEN guidance and output documentation. Neither explains when to call play_connect4 vs move_connect4, what the return string format is (board state, game status, etc.), or what errors the LLM should expect. Per pattern:tool-description, descriptions must state WHAT, WHEN, and prerequisites.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 38 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No error recovery guidance. move_connect4 returns 'Game hasn't started yet' if game is None, but tool description does not mention this error path or advise LLM to call play_connect4 first. Per pattern:recovery-guide, error responses must tell the LLM what to do next.
Output structure undocumented. play_connect4 and move_connect4 return string representations of board state, but no schema describes the format (is it ASCII art? CSV? JSON?). Per pattern:response-shaper, output schemas must be documented so LLMs can parse results.
Game state management is stateful (lifespan context) but this is not transparent in tool descriptions. LLMs cannot infer that play_connect4 resets global state or that move_connect4 modifies shared game object. Per pattern:tool, tools should be stateless or explicitly document side effects.
No idempotency guarantee. play_connect4 description says 'Can also restart game...' but does not guarantee that calling it twice produces identical state. Agents retry on failures, non-idempotent tools risk duplicate side effects. Per pattern:idempotent-operation.