Loading ToolBench
A web application for evaluating MCP (Model Context Protocol) server implementations with agent orchestration, run management, and performance analytics
mcpquick exposes 6 tools via FastAPI HTTP endpoints. All tools have descriptions (10-100 chars), but most are generic and lack LLM-optimization guidance. Input schemas are present for all tools, but parameter descriptions are minimal or missing entirely. The 'create_run' tool has the most complex schema (13 params) but lacks clarity on mutual exclusivity, defaults, and validation rules. No tool has documented output schemas. Error handling is absent, no recovery guidance, no categorization of retryable vs fatal errors. Tool naming is reasonable (verb_noun pattern) but descriptions do not explain WHEN to use each tool or what distinguishes similar ones (e.g., login vs me). Security concern: 'create_run' accepts 'api_key' as a parameter, violating secret-injection pattern.
Create a run record and enqueue asynchronous sandbox execution.
Return aggregate metrics used by the frontend dashboard view.
Lightweight health check that validates DB dependency wiring.
Authenticate with email/password and return a signed bearer token.
Return the authenticated user profile.
Register a new user and return an access token plus profile payload.
API key exposed as tool parameter in 'create_run'. Secrets must use server-side injection via environment variables, not tool parameters. Agent traces log all parameters, this leaks credentials.
'create_run' has 13 parameters with minimal descriptions. No documentation of mutual exclusivity (e.g., mcp_repos vs external_mcp_url), defaults, or validation rules. LLMs cannot infer parameter relationships.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. E.g., does 'register' return user_id? Does 'create_run' return run_id for polling?
No error handling or recovery guidance. Tools do not indicate which errors are retryable, which require user input, or which are fatal. LLMs receive raw HTTP errors with no actionable next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Parameter descriptions are generic or missing. E.g., 'mcp_failure_policy' has no enum values or explanation of valid options. 'max_steps' has no min/max bounds. LLMs must guess valid inputs.