Loading ToolBench
MCP sever for fetching latest Keyclaok documentation in LLM friendly format
Single tool with adequate naming and basic schema, but critical gaps in parameter descriptions, output documentation, and error handling. The tool name 'get-keycloak-release-notes' correctly starts with a verb and is clear, but the parameter description is minimal (only format examples provided). Output structure is not documented for LLMs. Error handling returns generic failure messages without recovery guidance. Resources feature is present but similarly underdocumented.
Get Keycloak's Release Notes with New Features and/or Breaking Changes
Parameter description lacks constraint detail. 'Keycloak version in format [major]_[minor]_[patch]' provides format but no guidance on valid version ranges, whether pre-release versions are accepted, or what happens with invalid formats.
Output schema is not documented. LLMs cannot infer the structure of returned content. The tool returns {content: [{text, type, name}]} but this is not explicitly described anywhere.
Error handling is non-actionable. The catch block returns 'Failed to fetch {version} Release Notes' without guidance on what the LLM should do next. Should it retry? Try a different version? Search for alternatives? No recovery path is offered.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 55 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 40 | - | v1 |
Resource handlers lack error context and descriptions. Six documentation resources are registered but have no descriptions explaining what each resource is, when to use it, or what format to expect. Generic error handling returns only the fetch error message.
Parameter allows any string without validation. A version string like 'invalid_version' or '999_999_999' will be accepted and sent to the GitHub API, returning a 404. The tool should validate format early and return clear error messages like 'Invalid version format: must be [major]_[minor]_[patch] (e.g., 26_2_0)'.