Loading ToolBench
MCP server for Keycloak Admin API - Manage Keycloak resources via MCP protocol
Scoring was not performed
Tool descriptions are below LLM-optimized length (15-20 chars vs 50-200 char baseline). 'List realm-level roles that are mapped to a client scope's scope' and 'Add realm-level roles to a client scope's scope' lack context for when to use these tools vs alternatives, prerequisites, and side effects.
No output schema documentation. Tools return Vec<RoleRepresentation> and (), but LLM context lacks field definitions for RoleRepresentation. Without documented return structure, agents cannot compose downstream tool calls or extract needed data reliably.
Parameter descriptions lack constraint details. 'id' is described as 'The client scope UUID' but does not explain format validation, length, or character restrictions. 'realm' is described as 'The realm name (not id!)' but could clarify case sensitivity, allowed characters, and validation rules.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 22 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 57 | 2024-11-05+ | v1 |
No error handling or recovery guidance in tool descriptions. Tools provide no guidance on what happens if realm does not exist, scope does not exist, roles are invalid, or permissions are denied. Agents cannot self-correct on failures.
Tool names are compound noun-heavy (e.g. 'client_scope_scope_mappings_realm_list') and 26-33 chars long, exceeding the p90 baseline of 27 chars. Names are difficult to parse visually and may cause LLM confusion when many tools share similar prefixes.
RoleRepresentation input parameter lacks inline schema documentation. The roles array contains 'Array of roles to add to the scope' with items marked as 'RoleRepresentation' but no field definitions provided to the LLM. LLM must infer required fields (id, name) without explicit guidance.