Loading ToolBench
Local MCP server for Qlik Cloud admin and governance operations
Single tool with adequate schema but weak description. The qlikcloud_login tool has a complete input schema with 4 parameters (force, prompt_login, open_browser, timeout), all with types and descriptions. However, the tool description is only 119 characters and lacks critical context: it does not explain WHEN to call it, what prerequisites exist, or what the returned 'summary dict' contains. The description does not state that this is a stateful operation with side effects (browser sign-in). No output schema is documented. The server has only 1 tool, limiting composition assessment.
Run the browser sign-in. Returns a summary dict with status, user_name, user_id, and scopes.
Output schema not documented. Tool returns 'summary dict with status, user_name, user_id, and scopes' but no structured schema is provided. LLMs cannot plan downstream calls or extract fields reliably.
Tool description lacks context for LLM selection. At 119 chars, it omits WHEN to call this tool, what prerequisites exist (e.g., is a browser required?), and whether it is idempotent. LLMs cannot determine if this is the right tool without explicit guidance.
Tool implementation not visible in source code. Only the tool registration and schema are shown; the actual handler in qlik_admin_mcp/auth/session.py is not provided.
Parameter 'timeout' lacks constraints. No minimum/maximum specified. Unbounded numeric parameters let LLMs pass absurd values (e.g., 999999 seconds) that break APIs or cause hangs.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 38 | <=2025-11-25 | v2 |
No error handling guidance. Tool description does not explain what errors are possible (network timeout, invalid credentials, browser unavailable) or how the LLM should recover. Agents cannot self-correct on failure.