Loading ToolBench
MCP server for Serverpod that provides tools to ask questions about Serverpod documentation, list guides, and retrieve guide content via the Starguide API.
The server has three tools with explicit schemas and descriptions, but exhibits significant quality gaps. All tools have basic descriptions (91-159 chars, within baseline 34-392 range) and proper input schemas. However, parameter documentation is missing, the Schema.string() and Schema.object() calls lack parameter descriptions. Output schemas are entirely undocumented. Error handling is minimal (ask-docs returns generic failure messages; get-guide returns plain text errors). Tools operate on a simple documentation/guide domain with no complexity, reducing composition risks but also limiting production applicability. The naming convention is clear and action-oriented (ask-, list-, get-), but parameter descriptions are absent, violating pattern:tool-description which requires every parameter to have a non-empty description. Overall quality is below baseline for production tools.
Ask any question about Serverpod. The answer will be based on the Serverpod documentation and answered GitHub discussions.
Retrieve a Serverpod guide by its URI.
List all Serverpod guides with their URI, name, and description.
Missing parameter descriptions. Schema.string(description: 'The question to ask about Serverpod') is present, but this approach does not establish full compliance with pattern:tool-description. All parameters lack the actionable context LLMs need (e.g., format, constraints, valid range). Example: 'uri' parameter in get-guide has only 'The URI of the resource to fetch', no indication of format, regex pattern, or example.
Output schemas entirely undocumented. Tools return CallToolResult with TextContent(text: ...), but the structure and fields of the returned text (e.g., for list-guides, a JSON array; for ask-docs, a free-form string) are not formally documented. LLMs cannot reliably parse or chain results without explicit output schemas.
Minimal error handling and no recovery guidance. ask-docs returns instructional text ('Please inform the user...') rather than structured error responses that guide the LLM on retry steps or alternatives. get-guide returns plain text errors ('Resource not found: $uri') with no suggestion of available resources or recovery paths.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | <=2025-11-25 | v2 |
ask-docs tool requires external Gemini API key configuration. If geminiApiKey is null, the tool returns a message instructing the user to be informed rather than a structured error. This violates pattern:secret-injection, secrets should never be exposed as parameters, but the entire tool is gated on external configuration without clear error classification or recovery path.
list-guides returns raw JSON string of guide objects. While functional, this is unstructured text (TextContent(text: jsonEncode(list))). Returning structured CallToolResult with typed result fields would enable LLMs to extract and chain guide URIs more reliably.
ask-docs tool description does not clarify that it depends on Gemini API key configuration or what happens if the key is missing. No indication of prerequisites or failure modes, violating pattern:tool-description guidance to 'state WHAT the tool does, WHEN to use it, and any prerequisites.'