Loading ToolBench
ReqRefine is an MCP conversational service that enhances requirement gathering through strategic questioning. It guides users to reveal comprehensive needs, uncovers implicit requirements, and transforms dialogue into structured specifications. ReqRefine turns requirements collection from a one-side interview into a collaborative discovery process.
The server defines 1 tool with explicit schema and registration. However, the tool has critical gaps in parameter documentation and schema completeness. The tool name is action-oriented but parameters lack individual descriptions required by the rubric. Output schema is not documented. The server exposes prompts and resources but these features do not compensate for weak tool definition quality. The tool description is minimal (18 chars) and fails to indicate when to use it or what it returns. Error handling in the tool implementation (src/req_refine/server.py lines 72-88) is present but does not guide LLM recovery, it raises ValueError without actionable guidance. This server is in the D range due to incomplete parameter documentation and absence of output schema definition.
Add a new note
Parameters lack individual descriptions in schema. JSON Schema for 'name' and 'content' properties has no 'description' field. Rubric: 'Every parameter needs a description explaining what it controls' (pattern:tool-description). LLMs cannot infer whether 'name' means a note title, filename, or identifier, or whether 'content' is plaintext, markdown, or structured data.
Tool description is 18 characters ('Add a new note'). Violates pattern:tool-description.
No output schema documented. The tool returns TextContent with hardcoded success message, but the schema definition (list_tools handler lines 53-66) does not declare a return type or response structure. LLMs cannot plan subsequent actions without knowing what fields are returned. Violates pattern:tool (D section: 'Document the output schema').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 36 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Error handling does not provide recovery guidance. The call_tool handler (lines 72-88) raises ValueError with generic messages like 'Missing arguments' or 'Unknown tool'. Per pattern:recovery-guide, errors should tell the LLM what to do next. Example: 'Missing arguments, required: name (string, max 100 chars), content (string, max 5000 chars)'.
Input schema lacks constraints. Parameters have no minLength, maxLength, pattern, or format constraints. This invites LLMs to pass absurd values (e.g., a 1GB note name). Violates pattern:constrained-input.
Tool modifies state (updates global notes dict) but description does not declare this. Per pattern:command-tool, 'If the tool modifies state (creates, updates, deletes, sends), the description must say so.' The current description 'Add a new note' does not signal that this is a side-effecting operation.